MEMORANDUM December 19, 2014 


TO: 


Board Members 


FROM: Terry B. Grier, Ed.D. 


Superintendent of Schools 


CONTACT: Carla Stevens, 713-556-6700 


SUBJECT: THINK THROUGH MATH IN HISD, 2013-2014 


Think Through Math (TTM) is an online, adaptive mathematics program designed to supplement 
classroom instruction. The program is funded by the Texas Education Agency for students in 
grades 3-8, and schools have the option to purchase the program for students in other grades. 
This report compares the STAAR mathematics achievement of HISD students in grades 3-8 who 
enrolled in the TTM program in 2013-2014 with the results of similar students in the same grades 
who did not use TTM. Three treatment groups of students were considered: all TTM users in 


the 


district, TTM users in 2013-2014 priority and focus schools, and TTM users who took the 


June 2014 retest of the STAAR mathematics test. 


Some of the highlights are as follows: 


Students in every grade level in HISD used TTM in 2013-2014 and 95 percent of the TTM 
users were in grades 3-8. 29,359 TTM users and 25,718 non-TTM users met the criteria for 
inclusion in this study, for a total of 55,077 students included in the analyses. 

Propensity score analyses were used to compare the mathematics achievement of TTM users 
matched with non-TTM users. Results indicated that HISD students who used TTM had 
significantly higher 2014 STAAR scale scores than did matched students who did not enroll 
in the program. 

TTM users at priority and focus schools who were matched with non-TTM users at the same 
schools were also found to have obtained significantly higher 2014 STAAR mathematics scale 
scores than non-TTM users had. 

For students who took the June 2014 STAAR retest, however, there were no significant 
differences in scale scores earned by TTM users when compared with the results of matched 
non-TTM users. 

Regression analyses indicated that for most groups of students, passing a higher number of 
TTM lessons, particularly passing a higher number of target TTM lessons, predicted a higher 
2014 STAAR mathematics scale score. Effect sizes for the significant relationships were 
small, with the exception of a moderately strong relationship between the number of TTM 
target lessons passed and 2014 STAAR mathematics scale scores for students who met the 
phase-in 1, satisfactory standard on the 2013 STAAR mathematics assessment. 


Should you have any further questions, please contact Carla Stevens in Research and 
Accountability at 713-556-6700. 
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2013-2014 


Executive Summary 


Evaluation Description 

Think Through Math (TTM) is an online, adaptive mathematics program designed to supplement classroom 
instruction in mathematics. In 2013-2014, the program was funded by the Texas Success Initiative through 
the Texas Education Agency and was free to students in grades three through eight, though additional 
services could be purchased by campuses to serve students through high school Algebra |. This study was 
designed to investigate the impact of the Think Through Math program on the State of Texas Assessment 
of Academic Readiness (STAAR) mathematics achievement of HISD students who used the program in 
grades three through eight. Three treatment groups of students were considered: all identified TTM users 
in the district, TTM users in 2013-2014 priority and focus schools, and TTM users who took the June 2014 
retest of the STAAR mathematics test. 


Highlights 
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A total of 51,863 HISD students, from every grade level, used TTM and could be matched with 
demographic data in 2013-2014. From the group of TTM users, 29,359 took both the spring 2013 and 
2014 STAAR mathematics assessments and were included in this analysis. A group of 25,718 HISD 
students who took the same exams but did not use TTM were included for comparisons of 
demographics and academic performance, making a total of 55,077 students whose performance was 
analyzed in this report. 


While 53 percent of students in the analysis used TTM, 66 percent of students in priority schools and 
56 percent of students who did not meet the phase-in 1 standard on the spring 2013 STAAR 
mathematics assessment did, indicating that TTM was used at higher rates with groups of students 
who needed academic support than it was used with other groups of students. 


A propensity score nearest neighbor matching analysis, in which spring 2014 STAAR mathematics 
scale scores were compared based on matching students on key variables, indicated that HISD 
students who used TTM had significantly higher 2014 STAAR mathematics scale scores than did 
matched students who did not enroll in the program. The same result was obtained when comparing 
spring 2014 STAAR mathematics scale scores for matched students at priority and focus schools. For 
students who retook the 2014 STAAR mathematics assessment in June, however, there were no 
significant differences in the scale scores earned by TTM users when compared with the scale scores 
earned by matched non-TTM users. 


Regression analyses were conducted to predict spring 2014 STAAR mathematics scale scores. 
Analyses were completed for all students in HISD and students in priority and focus schools, with 
students grouped based on spring 2013 STAAR mathematics results. For the majority of groups, the 
numbers of TTM lessons were significant predictors of 2014 STAAR mathematics scale scores. With 
some exceptions, the relationships were positive, i.e., a higher number of lessons predicted a higher 
scale score, however, the effect sizes, the magnitude of the differences, for the significant relationships 


were generally small. Only one effect size was moderate, that for the relationship between the number 
of target TTM lessons passed and spring 2014 STAAR mathematics scale scores for TTM users in 
HISD who met the 2013 STAAR mathematics phase-in 1, satisfactory standard. 


Recommendations 

e TTM data would be more accessible and reliable if data were associated with a single unique identifying 
number, such as the TSDS, rather than based on user names. Further, it would be helpful if TTM 
supported a system that allowed data to be modified without loss of information, and collected and 
accessible for more than a single academic year. 


e In this study, TTM was generally shown to be an effective academic support for HISD students in 
increasing their scale scores on the STAAR mathematics assessment. It is recommended that the 
program continue to be made accessible to schools and teachers to use as a supplement to regular 
instruction. 


e The number of TTM lessons a student passed and the number of target TTM lessons a student passed 
were more consistently associated with higher STAAR mathematics scale scores than the number of 
TTM lessons a student completed was. It is therefore recommended that teachers and monitors 
consider emphasizing students passing lessons over students simply completing them. 


Administrative Response 

Think Through Math, a supplemental program funded for all grade 3-8 students by the Texas Success 
Initiative, has been well received by both teachers and students in HISD so we have continued to 
recommend and support its use in the district. 


Recognizing that usage alone will not always have a positive result, we recommend that students not just 
complete lessons but rather that students pass the lessons they attempt. Students are encouraged to pass 
at least 30 target, or on grade-level, lessons prior to the administration of the STAAR-Mathematics. Think 
Through Math captures the idea in their catchphrase, “Think 30,” which can be heard and seen in many 3— 
8 classrooms in the district. 


We continue to recommend that teachers take an active role in the program by monitoring student use, 
utilizing the motivational features of the program, and using the various reports to review student progress 
and plan classroom instruction. In addition to grade-level lessons, the program offers lessons that 
remediate and scaffold the mathematics development of students with diverse needs. Teachers have 
significant control over the lessons students attempt in Think Through Math and are encouraged to use the 
program to individualize supplemental instruction as needed. 


A follow-up evaluation might consider the effect of Think Through Math on the various sub-populations we 
serve in our district. Also, anecdotal reports from campuses have been positive for high achieving students, 
and it is possible that a measure other than STAAR-Math would show a positive effect for high achieving 
students. The evaluation might also consider students’ feelings and perceptions about mathematics 
because many teachers report that students enjoy using Think Through Math. 


We continue working to build intentional usage of the program so that more HISD students can benefit and 


grow as mathematicians. To make data more accessible and student records more accurate, we began 
automated data provisioning for the 2014-2015 school year so that all student and teacher accounts were 
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set up based on Chancery. In the past, teachers/campuses were responsible for manually setting up 
accounts, which led to inaccuracies in student data and multiple student accounts. We have also requested 
that Think Through Math make historical student data reports available online so that teachers, 
administrators, and district-level personnel can track student growth over time. 
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Introduction 


Think Through Math (TTM) is an adaptive, web-based math enrichment program funded for students in 
grades three through eight through the Texas Success Initiative provided by the Texas Education Agency. 
Since the lessons are online, students can access the program both at school and during after-school hours 
free of charge. The program is also available for campuses to purchase for students in high school who 
are studying mathematics through Algebra I. 


Typically, a student takes a placement test at the beginning of a school year and, based on the results, is 
placed on a grade-level “pathway” which consists of approximately 30 target lessons. Remediation lessons 
are inserted on a student’s pathway if needed, and students can begin a new pathway when they finish the 
grade level assigned. Teachers have significant control on the presentation of the curriculum and can 
reorder, add, and remove lessons from a student’s pathway to meet classroom and/or student needs. The 
curriculum is designed to supplement rather than replace classroom teaching. 


A notable element of the program is the student motivators built into the lessons. Students earn points for 
all the actions they take in the program. Points can be used in a variety of ways, including to contribute to 
a class reward such as a pizza party or classroom supplies, to make a cash contribution to the student’s 
choice of a variety of charities, or to enhance a student's online TTM avatar. Students can earn points even 
for answering questions randomly, so teacher monitoring of use of the program is crucial. Teachers have 
ready access to information about each student's progress and can change a student’s pathway or send a 
personal note to a student instantly. 


According to the TTM website, “students who complete at least 30 lessons in their targeted pathway are 
proven to make measureable gains on standardized tests” (TTM, n.d., “Common Core Ready”) and users 
were encouraged to “Think 30” to increase their test scores. The statement is largely based on research 
done between 1991 and 1999 with 632 junior high and high school students learning 9' grade algebra 
using a product similar to Think Through Math for an average of 15—20 contact hours during the academic 
year (Meyer, et al., 2000). Further documentation is provided through more recent anecdotal information 
and case studies, which are available in local newspapers and on the Think Through Math website. This 
report provides academic performance and demographic information for HISD students who used the 
program in 2013-2014 and compares the results with those of similar HISD students who did not use the 
program. Performance is analyzed for all students as a group, and then disaggregated by students’ spring 
2013 STAAR mathematics performance and their accomplishments in the TTM program. In addition, the 
performance of students in 2013-2014 priority and focus schools is considered, as is the performance of 
students who did not successfully meet the phase-in 1 satisfactory standard on the spring 2014 STAAR 
mathematics assessment and retook the exam in June 2014. 


Methods 


Data Collection and Analysis 


e =Think Through Math achievement data came from TTM. A first draw was made for student work done 
by April 21, 2014, to document student achievement before the spring administration of the STAAR 
Mathematics exam, and the second was drawn for TTM work done by June 23, 2014, to measure any 
further progress in TTM before the STAAR retest. 
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e Demographic data were drawn from PEIMS Fall Resubmission files. 


e Data on STAAR performance came from data files from the Texas Education Agency. Scored versions 
of the regular STAAR and STAAR L (the linguistically accommodated test for English language 
learners) were used for the analyses. Performance of students on the spring 2013 administration of 
the mathematics subject test was used to analyze performance of the same students on the spring 
2014 and, for those who retested, the June 2014 administration of the mathematics subject test. 


e One student, a TTM user, who had met the phase-in 1 standard on the spring 2014 STAAR 
mathematics exam, also took the retest in June 2014. This student’s score was not included in the 
analysis of the achievement of TTM users on the June retest, leaving a total of 2,789 students for the 
analysis of the performance of June 2014 retesters. 


e Propensity score nearest neighbor matching, used to make causal inferences based on observational 
data (Cohen, 1988), was used to compare the performance of students who took both the spring 2013 
and 2014 STAAR mathematics assessments and used TTM in 2013-2014 with the performance of 
other students in the district who took the same tests. Students in grades three through eight were 
compared within three groups: all students in HISD; students in priority and focus schools; and students 
who retook the STAAR mathematics exam in June 2014. Students in each group were matched on 
spring 2013 STAAR mathematics scale score and 2014 enrollment in a priority or focus school, grade 
level, gender, economic disadvantage, limited English proficiency (LEP) status, and race/ethnicity. In 
each case, a probit regression analysis indicated significant differences between the groups initially. 
For the propensity score matching comparison within HISD, all 55,077 students were included and 
balance was achieved on each of the matching variables, i.e., after the matches were made, there were 
no significant differences between the groups on any of the matching variables. For the comparison 
within priority and focus schools, all 12,792 students were included; balance was not achieved for 
economic disadvantage or for the ethnic category American Indian. For the comparison of students 
who retested in June, all 2,789 students were used and balance was achieved on each matching 
variable. 


e In order to establish the performance of students based on the number of TTM lessons they used, 
regression analyses were run for the relationship between spring 2014 STAAR mathematics scale 
scores and the number of lessons TTM users completed, the number of lessons they passed, and the 
number of target lessons they passed. Completed lessons were those that students finished but did 
not necessarily pass. Completed lessons included target lessons, specifically aligned to the grade level 
curriculum, and remedial or administrative lessons, inserted to allow students to negotiate the program 
and to provide lessons on content students had not yet mastered. Passed lessons were those for which 
students were successful in a summative assessment, and could have been either target or other 
inserted lessons. Target lessons completed were those specifically designed to address grade level 
content on which students were successful in the summative assessment. Data for each category were 
provided by TTM. 


e Priority and focus schools were those identified by the Texas Education Agency for the 2013-2014 


academic year. Priority elementary and middle schools were in the lowest five percent of Title | schools 
academically, based on state reading and mathematics assessments, and focus schools were in the 
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lowest ten percent of Title | schools based on the widest gaps between student performance and 
established federal targets (Gonzalez Reynolds, 2013). 


e _Inthis report, numbers were rounded to the nearest whole number in the text, and to the nearest tenth 
in the tables. Numbers were rounded up if the next digit was 5 or higher and were not changed if the 
next digit was lower, so 11.49 was recorded as 11.5 in a table and 11 in the text, while 11.50 was 
recorded as 11.5 in the table and 12 in the text. 


e For percentages associated with demographics, percentages in the text are reported as percentage of 
TTM users and non-TTM users who share a given characteristic. In the tables, percentages are 
reported as a distribution of a given characteristic within the group of TTM users or the group of non- 
TTM users. Percentages reported in the text can be reproduced by using the corresponding table of 
demographics selecting a single demographic characteristic, and dividing the number of students in the 
chosen group, TTM user or non-TTM user, by the sum of the number of TTM users and non-TTM users 
who share the same characteristic. 


Data Limitations 
For a complete description of data limitations, see Appendix A, page 48. 


Results 


How many HISD students used TTM in 2013-2014 and how did they compare with their peers in the 
district? 


e A total of 51,863 HISD students were identified as TTM users and were matched with demographic 
data and in 2013-2014. As detailed in Table 1, (page 49), nine percent of TTM users attended a 2013-— 
2014 priority school, 19 percent attended a 2013-2014 focus school, and the remainder attended a 
non-prioritized school. Also in Table 1 (page 49) and illustrated in Figure 1 (page 7), TTM users were 
represented in every grade level, as the registration process for TTM facilitated small numbers of 
students in lower grades enrolling in the program. The majority of HISD TTM users, 68 percent, 
attended elementary grades three through five, and 27 percent attended middle school grades six 
through eight, for a total of 95 percent of TTM users attending grades three through eight. 
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Figure 1. Percentage of HISD students who used TTM by grade level, 2013-2014 
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Note: Percentages may not equal 100 due to rounding 
Sources: PEIMS Fall Resubmission and TTM files 


e Ofthe HISD students who used TTM, 29,359 took both the spring 2013 and 2014 STAAR mathematics 
assessments. Another 25,718 HISD students took the same exams at the same times but did not use 
TTM in 2013-2014. Demographic information for these students, including numbers that allow 
calculation of the results in Figures 2-6 (pages 8-12) can be found in Table 2 (page 50). Shown in 
Figure 2, 53 percent of students who took both the spring 2013 and 2014 STAAR mathematics 
assessments used TTM in 2013-2014, while 47 percent did not. 


Figure 2. Percentage of TTM users and non-TTM users among students who took both 
the spring 2013 and 2014 STAAR mathematics assessments, 2013-2014 
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Sources: PEIMS Fall Resubmission and TTM files 
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e Seventy-six (76) percent of TTM users who took both STAAR mathematics assessments attended 
schools that were not prioritized by the state of Texas. Shown in Figure 3, of students who attended 
priority schools and took both STAAR mathematics tests, 66 percent used TTM and 34 percent did not, 
while at focus schools and at non-prioritized schools, 53 percent used TTM and 47 percent did not. 
Since 53 percent of all students who took both STAAR mathematics tests used TTM (see Figure 2, 
page 7), TTM usage was proportionate in focus and non-prioritized schools, but was used more broadly 
at priority schools. 


Figure 3. Percentage of TTM users and non-TTM users among students who took both the 
spring 2013 and 2014 STAAR mathematics assessments, 
by 2013-2014 school prioritization 
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Illustrated in Figure 4, in the elementary grades three through five, the majority of students who took 


both the spring 2013 and 2014 STAAR mathematics exams were enrolled in TTM, while in the middle 
school grades six through eight, a majority of students who tested both years were not enrolled in the 


program. 


Figure 4. Percentage of TTM users and non-TTM users among students who took both the 
spring 2013 and 2014 STAAR mathematics assessments, by 2013-2014 grade level 
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Grade3 Grade4 Grade5 Grade6 Grade7 Grade8 


STTM Users &Non-TTM Users 


Students in grade three took the grade three level STAAR mathematics test in 
both 2013 and 2014, indicating that they had been retained in third grade. 
Percentages may not equal 100 due to rounding. 

PEIMS Fall Resubmission and TTM files 


e Fifty-three (53) percent of the females and 54 percent of the males who took both the spring 2013 and 
2014 STAAR mathematics assessments used TTM. Since 53 percent of all the students were TTM 
users, gender was distributed relatively evenly within the groups of TTM and non-TTM users. Shown 
in Figure 5, compared with TTM usage in all schools (53 percent of students in all the schools used 
TTM) (See Figure 2, page 7), a smaller percentage of whites and Asian/Pacific Islanders used TTM, 
and a larger percentage of other students (American Indians and those with two or more race/ethnicity 
designations) used the program. 


Figure 5. Percentage of TTM users and non-TTM users among students who took both the 
spring 2013 and 2014 STAAR mathematics assessments, by gender and race/ethnicity 
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Sources: PEIMS Fall Resubmission and TTM files 
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e = Illustrated in Figure 6, half of the students who took both the spring 2013 and 2014 STAAR 
mathematics exams and who had no economic disadvantage used TTM and half did not, indicating 
that TTM was more broadly used among students in every category of economic disadvantage than 
among students who had no economic disadvantage. Grouped by English proficiency, a higher 
percentage of students who had limited English proficiency (LEP) used TTM than did students who had 
no limitations in English proficiency. 


Figure 6. Percentage of TTM users and non-TTM users among students who 
took both the spring 2013 and 2014 STAAR mathematics assessments, 
by economic disadvantage and English proficiency 
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Sources: PEIMS Fall Resubmission and TTM files 


e More detail on the demographics of students, both TTM users and non-TTM users, who took both the 
spring 2013 and 2014 STAAR mathematics assessments can be found in Table 2 (page 50). 
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e =Spring 2013 STAAR mathematics results for the 55,077 students who took both the spring 2013 and 
2014 STAAR mathematics exams are shown in Figure 7 and Table 3 (page 51). A lower percentage 
of TTM users than of non-TTM users met the spring 2013 phase-in 1, satisfactory standard but a slightly 
higher percentage met the advanced standard. 


Figure 7. Spring 2013 STAAR mathematics performance of students who took both 
the spring 2013 and 2014 STAAR mathematics assessments, by TTM useage 
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Sources: 2013 and 2014 STAAR mathematics files and TTM files 
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Spring 2014 STAAR mathematics results for the same students, those in HISD who took both the 2013 
and 2014 spring STAAR mathematics assessments, are shown in Figure 8 and Table 4 (page 51). A 
higher percentage of TTM users and also of non-TTM users met the phase-in 1, satisfactory standard 
in spring 2014 than in spring 2013. The gap between the percentage of TTM and non-TTM users who 
met the phase-in 1, satisfactory standard was reduced from four percentage points in 2013 to one 
percentage point in 2014, with TTM users attaining the lower percentage meeting the satisfactory 
standard both years. At the same time, the gap between the percentage of TTM and non-TTM users 
who earned the advanced standard increased from one percentage point to three percentage points, 
with TTM users attaining the higher percentage achieving the advanced standard both years. 


Figure 8. Spring 2014 STAAR mathematics performance of students who took 
both the spring 2013 and 2014 STAAR mathematics assessments, by TTM useage 
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Sources: 2013 and 2014 STAAR mathematics files and TTM files 


Analysis using propensity score nearest neighbor matching, in which spring 2014 STAAR mathematics 
scale scores were compared based on matching students on key variables which included spring 2013 
STAAR mathematics scale scores, enrollment in a priority or focus school, grade level, gender, 
economic disadvantage, LEP status, and race/ethnicity, indicated that students who used TTM had 
significantly higher spring 2014 STAAR mathematics scale scores than did matched students who did 
not enroll in the program. All 55,077 students in the sample were included in the analysis. Shown in 
Table 5 (page 51), students enrolled in TTM earned an average of 12 scale score points more than 
matched students not enrolled in the program earned. To put the 12 point difference in scale scores in 
context, while differences in the scale score cut-off points for achieving the phase-in 1 satisfactory 
standard on STAAR are different for each grade level from three to eight, a student must gain an 
average of 38 scale score points each year to meet the standard set for a year’s progress. While all 
students were expected to earn an average of 38 scale points in a year, TTM students earned an 
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average of 12 more scale points in 2013-2014 than non-TTM users earned. The difference in scale 
score points earned by TTM users compared with the scale score points earned by non-TTM users 
was significant at the p<.001 level. 


Based on the number of TTM lessons they completed in 2013-2014, how did HISD students who did 
not meet the spring 2013 STAAR mathematics phase-in 1, satisfactory standard perform on the 
spring 2014 STAAR mathematics assessment? 


e Of the 55,077 HISD students who took both the spring 2013 and 2014 STAAR mathematics 
assessments, 18,322 (33 percent) did not meet the phase-in 1, satisfactory standard on the spring 2013 
STAAR Mathematics assessment. Of those students, 10,227 (56 percent) were enrolled in TTM in the 
2013-2014 school year. As shown in Figure 9 and detailed in Table 6 (page 52), the majority of 
students who did not meet the phase-in 1, satisfactory standard in 2013 also failed to meet the 
satisfactory standard in 2014, however 37 percent of TTM users met the phase-in 1, satisfactory 
standard on the spring 2014 exam while a slightly lower percentage, 36 percent, of non-TTM users did. 


Figure 9. Spring 2014 STAAR mathematics performance of students who took both 
the spring 2013 and 2014 STAAR mathematics assessments and did not meet 
the phase-in 1, satisfactory standard on the spring 2013 exam 
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As shown in Figure 10, the majority of TTM users who did not meet the spring 2013 STAAR 
mathematics standard completed and passed 1-14 TTM lessons in 2013-2014, 61 percent and 84 
percent of students, respectively. Fourteen (14) percent of the TTM users completed 30 or more TTM 
lessons, the recommended goal set by TTM, while two percent of users passed 30 or more of the TTM 
lessons they completed. Only 0.2 percent of the users passed thirty or more TTM target lessons. 


Figure 10. Percentage of TTM users who did not meet the spring 2013 STAAR mathematics 
phase-in 1, satisfactory standard, by number of TTM lessons completed, number of 
TTM lessons passed, and number of target TTM lessons passed, 2013-2014 
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The 2014 STAAR mathematics results for HISD students who failed to meet the phase-in 1, satisfactory 
standard on the spring 2013 STAAR mathematics assessment are illustrated in Figure 11 (page 16) 
and detailed in Table 6 (page 52) for non-TTM users and Table 7 (page 52) for TTM users. Within this 
group, a higher percentage of TTM users met the 2014 STAAR mathematics phase-in 1 satisfactory 
standard than did non-TTM users (the last column in the figure), except for TTM users who completed 
1-14 TTM lessons. Forty-one (41) percent of the students who did not meet the 2013 standard and 
completed thirty or more TTM lessons in 2013-2014 passed the spring 2014 STAAR mathematics 
exam with the phase-in 1, satisfactory standard, and one percent of them achieved the advanced 
standard. For comparison, 37 percent of non-TTM users met the phase-in 1 satisfactory standard and 
one percent achieved the advanced standard. 
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Figure 11. Percentage of students who achieved satisfactory and advanced ratings 
on the spring 2014 STAAR mathematics assessment after not meeting the satisfactory 
standard on the spring 2013 STAAR mathematics test, by number of TTM lessons 
completed and compared with HISD students who did not enroll in TTM 
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Sources: 2013 and 2014 STAAR mathematics files and TTM files 


Three regression analyses were conducted, one with the number of TTM lessons completed as the 
predictor, the second with the number of TTM lessons passed as the predictor, and the third with the 
number of TTM target lessons passed as the predictor. The criterion variable for all three analyses 
was the spring 2014 STAAR mathematics scale score. As shown in Table 8 (page 53), two of the three 
independent variables were significant predictors of 2014 STAAR mathematics scale scores, the 
number of TTM lessons passed and the number of TTM target lessons passed. In both cases, more 
TTM lessons were associated with higher spring 2014 STAAR mathematics scale scores. The 
magnitude of the difference (the effect size) between the sets of 2014 STAAR mathematics scale 
scores was small for both relationships. 


The average numbers of TTM lessons used by students who did not meet the phase-in 1, satisfactory 
standard in 2013 but who successfully met the satisfactory standard and/or achieved the advanced 
standard on the spring 2014 STAAR mathematics exam are shown in Table 9 (page 54). Students 
who successfully met the 2014 satisfactory standard completed an average of 18 TTM lessons, passed 
an average of seven, and passed an average of two target TTM lessons. Those who also met the 
advanced standard had higher average numbers for each category, 24 completed lessons, 12 passed 
lessons, and five passed target lessons. The standard deviation associated with each of the averages 
was high, indicating a wide distribution of numbers of lessons was associated with the groups of 
successful students. 
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Based on the number of TTM lessons they completed in 2013-2014, how did HISD students who 
met the 2013 STAAR mathematics phase-in 1, satisfactory standard perform on the spring 2014 
STAAR mathematics assessment? 


e Of the 55,077 HISD students who took both the spring 2013 and 2014 STAAR mathematics 
assessments, 36,755 (67 percent) met the phase-in 1, satisfactory standard on the spring 2013 STAAR 
mathematics test. Of these students, 19,132 (52 percent) used TTM and 17,623 (48 percent) did not. 
Shown in Figure 12 and Table 10 (page 54), almost 90 percent of students who met the phase-in 1, 
satisfactory standard in spring 2013 did so again in spring 2014. Notably, a higher percentage of TTM 
users than non-TTM users both met the phase-in 1, satisfactory standard and met the advanced 
standard on the spring 2014 STAAR mathematics exam. 


Figure 12. Spring 2014 STAAR mathematics performance of students who took 
both the spring 2013 and 2014 STAAR mathematics assessments and met 
the phase-in 1, satisfactory standard on the spring 2013 exam 
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e Pictured in Figure 13, the majority of TTM users who met the phase-in 1, satisfactory standard on the 
spring 2013 STAAR mathematics completed and passed from 1-14 TTM lessons and passed 1-14 
target TTM lessons. Seventeen (17) percent completed at least 30 TTM lessons and three percent 
passed 30 or more target TTM lessons. The percentage of students in all three categories of 30 or 
more TTM lessons was higher than the percentage in the same categories for students who did not 
meet the 2013 phase-in 1, satisfactory standard (for comparison, see Figure 10, page 16). 


Figure 13. Percentage of TTM users who met the spring 2013 STAAR mathematics 
phase-in 1, satisfactory standard, by number of TTM lessons completed, 
number of TTM lessons passed, and number of 
target TTM lessons passed, 2013-2014 
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On the spring 2014 STAAR mathematics assessment, TTM users who completed 0-14 TTM lessons 
had the same or lower percentages of students meeting the phase-in 1, satisfactory standard but higher 
percentages meeting the advanced standard compared with students who did not use TTM (see Figure 
14 and Table 10, page 54, for non-TTM users and Table 11, page 54, for TTM users). Students who 
completed and passed 15 or more TTM lessons had higher percentages of students meeting both the 
satisfactory and advanced standards than did non-TTM users. The percentage of students who 
achieved the satisfactory and/or advanced standard on the spring 2014 STAAR mathematics test was 
higher for students who met the 2013 phase-in 1, satisfactory standard than it was for students who did 
not meet the standard, in all categories illustrated (for comparison, see Figure 11, page 16). 


Figure 14. Percentage of students who achieved satisfactory and advanced ratings 
on the spring 2014 STAAR mathematics assessment after meeting the satisfactory 
standard on the spring 2013 STAAR mathematics test, by number of TTM lessons 

completed and compared with HISD students who did not enroll in TTM 
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Regression analyses using the number of TTM lessons completed, the number of TTM lessons passed, 
and the number of target TTM lessons passed to predict spring 2014 STAAR mathematics scale scores 
are provided in Table 12 (page 55). All three categories of numbers of lessons were significant 
predictors of spring 2014 test scores, with higher numbers of TTM lessons associated with higher spring 
2014 STAAR mathematics scale scores. The effect sizes for the number of TTM lessons completed 
and the number of TTM lessons passed was small, but the effect size for the number of target TTM 
lessons passed was medium, indicating a greater magnitude in the difference in 2014 scale scores. 


The average number of lessons in each category for TTM users who successfully met the 2013 STAAR 
mathematics phase-in 1 satisfactory standard and also met the 2014 satisfactory standard and/or the 
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2014 advanced standard is provided in Table 13 (page 55). Students who met the satisfactory standard 
completed an average of 19 lessons, passed 12, and passed an average of six target TTM lessons; 
those who also met the advanced standard averaged more lessons in each category, 23 TTM lessons 
completed, 18 passed, and 12 target TTM lessons passed. 


e The average number of lessons in each category for TTM users who met the 2013 phase-in 1, 
satisfactory standard (Table 13, page 55) was higher than the complementary average for successful 
students who did not meet the 2013 phase-in 1 satisfactory standard (Table 9, page 53), with the 
exception of the average number of TTM lessons completed by students who were successful on the 
2013 STAAR mathematics test and who achieved the 2014 advanced standard. 


Based on the number of TTM lessons they completed in 2013-2014, how did HISD students who 
met the spring 2013 STAAR mathematics advanced standard perform on the spring 2014 STAAR 
mathematics assessment? 


e A total of 7,881 students (14 percent of those in the group of students who took both the spring 2013 
and 2014 STAAR mathematics exams) achieved the advanced standard on the 2013 STAAR 
mathematics test. Of these, 4,262 (54 percent) used TTM and the remainder did not. Illustrated in 
Figure 15 and detailed in Table 14 (page 56), 99 percent of both TTM users and non-TTM users 
achieved the phase-in 1, satisfactory standard on the spring 2014 STAAR mathematics test, and a 
majority of both groups again achieved the advanced standard, with the percentage of TTM users 
earning the advanced standard exceeding that of non-TTM users. 


Figure 15. Spring 2014 STAAR mathematics performance of students who took 
both the spring 2013 and 2014 STAAR mathematics assessments and met 
the advanced standard on the spring 2013 exam 
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e Shown in Figure 16, the majority of TTM users who earned the advanced standard on the spring 2013 
STAAR mathematics exam completed and passed 1-14 TTM lessons and also 1-14 target TTM 
lessons in 2013-2014. This group had larger percentages of students completing and passing 30 or 
more TTM lessons and also passing 30 or more target TTM lessons than did students who did not meet 
the advanced standard on the spring 2013 STAAR mathematics test (for comparison, see Figure 10, 
page 15, and Figure 13, page 18). 


Figure 16. Percentage of TTM users who met the 2013 STAAR mathematics phase-in 1, 
satisfactory standard, by number of TTM lessons completed, number of TTM lessons 
passed, and number of target TTM lessons passed, 2013-2014 
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When students who achieved the advanced standard on the spring 2013 STAAR mathematics 
assessment were grouped by the number of TTM lessons completed, either 99 or 100 percent in each 
category achieved the satisfactory standard on the spring 2014 STAAR mathematics exam. Ilustrated 
in Figure 17 and detailed in Table 14 (page 56) for non-TTM users and in Table 15 (page 56) for TTM 
users, TTM users who completed 1-14 TTM lessons had the same percentage of students achieving 
the advanced standard on the spring 2014 STAAR mathematics test as non-TTM users had, but TTM 
users who completed more lessons had higher percentages of students earning the satisfactory and 
advanced ratings in 2014 than non-TTM users had. 


Figure 17. Percentage of students who achieved satisfactory and advanced ratings 

on the spring 2014 STAAR mathematics assessment after meeting the advanced 

standard on the spring 2013 STAAR mathematics test, by number of TTM lessons 
completed and compared with HISD students who did not enroll in TTM 
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The results of regression analyses using numbers of lessons as predictor variables for students who 
earned the advanced standard on the 2013 STAAR mathematics assessment are reported in Table 16 
(page 57). The number of TTM lessons completed, the number of TTM lessons passed, and the 
number of target TTM lessons passed were all significant predictors of spring 2014 STAAR 
mathematics scale scores with higher numbers of lessons associated with higher spring 2014 STAAR 
mathematics scale scores. All the relationships were significant at the p<.001 level, but all the effect 
sizes were small. 


The average number of TTM lessons completed and passed, and the average number of target TTM 
lessons passed by students who achieved the advanced standard on the spring 2013 STAAR 
mathematics exam and were successful on the spring 2014 test are listed in Table 17 (page 57). 
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Students who achieved the advanced standard on both tests completed an average of 24 TTM lessons, 
close to but not surpassing the TTM recommendation of 30. They passed an average of 20 TTM 
lessons, and an average of 14 target TTM lessons. 


e The average number of TTM lessons completed and passed, as well as the average number of target 
TTM lessons completed by students who earned the advanced standard on the spring 2013 STAAR 
mathematics test exceeded the averages recorded for students who did not achieve that standard on 
the spring 2013 exam (for comparison, see Table 9, page 53, Table 13, page 55, and Table 17, page 
57). 


How many students in HISD priority and focus schools used TTM in 2013-2014 and how did they 
compare with their peers enrolled in priority and focus schools who did not use TTM? 


e Of the 51,863 students who used TTM and could be matched with demographic data in 2013-2014, 
14,901 of them, 29 percent of identifiable TTM users in HISD, attended an HISD priority or focus school. 
TTM users in priority and focus schools were found at every grade level except grade one. As shown 
in Figure 18, the majority of TTM users were enrolled in grades three through eight, the grade levels 
in which TTM lessons were funded by the state. 


Figure 18. Percentage of students enrolled in HISD priority and focus schools who used TTM, 
by grade level, 2013-2014 
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In HISD, 55,077 students took both the spring 2013 and 2014 STAAR mathematics assessments and 
were associated with demographic information. Of these students, 12,792 (23 percent) attended an 
HISD priority or focus school. Demographic information for these students, including numbers that 
allow calculation of the results in Figures 19-22 (pages 24—27) can be found in Table 19 (page 59). 
Illustrated in Figure 19, 66 percent of students in priority schools used TTM, and 53 percent of students 
in focus schools used the program. For comparison, 53 percent of all HISD students who took both 
the spring 2013 and 2014 STAAR mathematics exams used TTM in 2013-2014 (see Figure 2, page 
7), indicating that TTM was used by a higher percentage of students in priority schools than in other 
schools in HISD, including in focus schools. Overall, a higher percentage of students who attended a 
2013-2014 priority or focus school, 55 percent, used TTM than did all HISD students in the analysis, 
53 percent. 


Figure 19. Percentage of TTM users and non-TTM users among students who attended 
a 2013-2014 priority or focus school and took both the spring 2013 and 
2014 STAAR mathematics assessments 
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e More than 80 percent of third-, fourth-, and fifth-grade students in priority and focus schools who took 
both the spring 2013 and 2014 STAAR mathematics assessments used TTM in 2013-2014, while fewer 
than 50 percent of priority and focus school students in grades six through eight used the program the 
same year. More detail can be seen in Figure 20. 


Figure 20. Percentage of TTM users and non-TTM users among students who attended 


a 2013-2014 priority or focus school and took both the spring 2013 and 2014 STAAR 
mathematics assessments, by 2013-2014 grade level 
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e = Illustrated in Figure 21, 55 percent of females and 55 percent of males in priority and focus schools 
who took both the spring 2013 and 2014 STAAR mathematics exams used TTM. Since 55 percent of 
students in priority and focus schools used TTM, these percentages match those of the whole group, 
indicating that gender was proportionally represented in each group. By race/ethnicity, a higher 
percentage of African Americans and of other students (American Indians and those with two or more 
race/ethnicity designations) in priority and focus schools used TTM while a lower percentage of white 
and Asian/Pacific Islander students in the same schools used the program. 


Figure 21. Percentage of TTM users and non-TTM users among students who attended 


a 2013-2014 priority or focus school and took both the spring 2013 and 2014 STAAR 
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e The percentage of economically disadvantaged students who attended 2013-2014 priority and focus 
schools, took both the spring 2013 and 2014 STAAR mathematics assessments, and used TTM was 
generally the same as the percentage of TTM users in priority and focus schools. At the same time, 
the percentage of non-economically disadvantaged students who used TTM was lower than the 55 
percent figure for the sample of students from priority and focus schools (Figure 22). By LEP status, 
the percentage of LEP and non-LEP students using TTM generally matched the percentage of TTM 
users from priority and focus schools, indicating that LEP status was distributed relatively 
proportionately between the groups. 


Figure 22. Percentage of TTM users and non-TTM users among students who attended a 
2013-2014 priority or focus school and took both the spring 2013 and 2014 STAAR 
mathematics assessments, by economic disadvantage and English proficiency 
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e See Table 18 (page 58), for more detail on demographic characteristics of students in priority and focus 


schools associated with TTM usage and Table 19 (page 59), for a comparison of demographics for 
TTM and non-TTM users in the analyses. 
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e Academic performance on the spring 2013 STAAR mathematics assessment is shown in Figure 23 
and in Table 20 (page 60), for students who attended a 2013-2014 priority or focus school and took 
both the spring 2013 and 2014 STAAR mathematics tests. Fifty-six (56) percent of the priority and 
focus school students who used TTM in 2013-2104 did not meet the phase-in 1, satisfactory standard 
on the spring 2013 STAAR mathematics exam, while 51 percent of students in the same schools who 
did not use TTM had the same result. A higher percentage of TTM users, five percent, achieved the 
advanced standard than did non-TTM users, with four percent of students earning the advanced 
standard. The percentages of TTM and non-TTM users in priority and focus schools who failed to meet 
the 2013 STAAR mathematics satisfactory standard were higher than those of all TTM and non-TTM 
users in HISD (See Figure 7, page 12), and a lower percentage of students in priority and focus schools 
achieved the advanced standard on the same test. 


Figure 23. Spring 2013 STAAR mathematics performance of students who attended a 
2013-2014 priority or focus school and took both the spring 2013 and 2014 STAAR 
mathematics assessments, by TTM useage 
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Sources: 2013 and 2014 STAAR mathematics files, TEA priority and focus school 
identification files, and TTM files 
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The same priority and focus school students’ academic performance on the spring 2014 STAAR 
mathematics exam is illustrated in Figure 24 and detailed in Table 21 (page 60). A majority of both 
TTM and non-TTM users met the phase-in 1, satisfactory standard, with a slightly higher percentage of 
non-TTM users than TTM users meeting the satisfactory standard. However, between 2013 and 2014, 
the gap between the two groups closed by three percentage points. While the percentage of non-TTM 
users who achieved the advanced standard remained at four percent for both 2013 and 2014, the 
percentage of TTM users who earned the advanced standard rose from five percent on the 2013 test 
to seven percent in 2014. 


Figure 24. Spring 2014 STAAR mathematics performance of students who attended a 
2013-2014 priority or focus school and took both the spring 2013 and 2014 STAAR 
mathematics assessments, by TTM useage 
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A propensity score nearest neighbor matching analysis was done to compare spring 2014 STAAR 
mathematics scale scores between TTM users and non-TTM users who attended priority or focus 
schools and had taken both the spring 2013 and 2014 STAAR mathematics assessments. Students in 
the two groups were matched on spring 2013 STAAR mathematics scale scores, grade level, gender, 
economic disadvantage status, LEP status, and race/ethnicity. All 12,792 students in the sample were 
included in the analysis. The results, seen in Table 22 (page 61), indicated a significant difference in 
spring 2014 STAAR mathematics scale scores, p < .001, between TTM and non-TTM users, with TTM 
users earning an average of 16 scale score points more than non-TTM users earned. For a context for 
interpreting scale score points, please see the explanation in the bullet for propensity score matching 
on pages 13-14 of this report. 
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Based on the number of TTM lessons they completed in 2013-2014, how did HISD students who 
attended a 2013-2014 priority or focus school and who did not meet the spring 2013 STAAR 
mathematics phase-in 1, satisfactory standard perform on the spring 2014 STAAR mathematics 
assessment? 


e = Fifty-three (53) percent of the 6,837 students who attended 2013-2014 priority or focus schools and 
who took both the spring 2013 and 2014 STAAR mathematics assessments did not meet the phase-in 
1, satisfactory standard on the 2013 assessment. Shown in Figure 25 and detailed in Table 23 (page 
61), the majority of the students also did not meet the standard for the spring 2014 STAAR mathematics 
test. A slightly higher percentage of non-TTM users than TTM users met the 2014 phase-in 1, 
satisfactory standard, and, though numbers of students meeting the advanced standard were small, a 
few more TTM users (12) than non-TTM users (seven) met the advanced standard. 


Figure 25. Spring 2014 STAAR mathematics performance of students who attended a priority or 
focus school, took both the spring 2013 and 2014 STAAR mathematics assessments, and 
did not meet the phase-in 1, satisfactory standard on the 2013 exam 
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Sources: 2013 and 2014 STAAR mathematics files, TEA priority and focus school 
identification files, and TTM files 
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e The number of TTM lessons completed, the number of TTM lessons passed, and the number of target 
TTM lessons passed by TTM users in priority and focus schools who took both the spring 2013 and 
2014 STAAR mathematics assessments and did not meet the 2013 satisfactory standard are shown in 
Figure 26 and in Table 24 (page 62). The majority of students completed and passed 1-14 TTM 
lessons and passed zero target TTM lessons. Fourteen (14) percent completed 30 or more TTM 
lessons, as recommended by the TTM company, and one percent passed 30 or more lessons. 


Figure 26. Percentage of TTM users who attended a 2013-2014 priority or focus school and did 
not meet the 2013 STAAR mathematics phase-in 1, satisfactory standard, by number of TTM 
lessons completed, number of TTM lessons passed, and number of 
target TTM lessons passed, 2013-2014 
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e Shown in Figure 27 (page 32) and in Table 24 (page 62), the percentage of students who attended 
2013-2014 priority and focus schools and who failed to meet the 2013 STAAR mathematics phase-in 
1, satisfactory standard but who were successful on the spring 2014 exam generally rose slightly as 
the number of completed lessons rose, from 30 percent of students who completed no TTM lessons to 
33 percent of students who completed 30 or more TTM lessons. For comparison, 31 percent of non- 
TTM users who attended a priority or focus school and who did not meet the 2013 satisfactory standard 
succeeded in meeting the standard on the 2014 assessment (Figure 27, last column, page 32, and 
Table 23, page 61). 
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Figure 27. Percentage of students who attended a 2013-2014 priority or focus schools and 
achieved satisfactory and advanced ratings on the spring 2014 STAAR mathematics 
assessment after not meeting the satisfactory standard on the spring 2013 STAAR 
mathematics test, by number of TTM lessons completed and compared 
with students in priority and focus schools who did not enroll in TTM 
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Sources: 2013 and 2014 STAAR mathematics files, TEA priority and focus school 
identification files, and TTM files 


Regression analyses of the relationships between numbers of lessons and spring 2014 STAAR 
mathematics scale scores for students who atteded 2013-2014 priority and focus schools and who did 
not achieve the phase-in 1, satisfactory standard on the 2013 STAAR mathematics assessment are 
reported in Table 25, (page 63). The number of TTM lessons completed, the number of TTM lessons 
passed, and the number of target TTM lessons passed were all significant predictors of spring 2014 
STAAR mathematics scale scores. For the number of TTM lessons completed, the relationship was 
negative, meaning higher numbers of lessons were associated with lower spring 2014 STAAR 
mathematics scale scores. Though the relationship was significant, the magnitude of the relationship 
was very small. For the number of TTM lessons passed and the number of target TTM lessons passed, 
a positive relationship was found. A higher number of lessons predicted a higher 2014 STAAR 
mathematics scale score. The relationships were significant, but the effect sizes were small. 


The average number of TTM lessons completed, number of TTM lessons passed, and number of TTM 
target lessons passed for students who attended a priority or focus school and who did not meet the 
satisfactory standard on the spring 2013 STAAR mathematics test but who were successful on the 
spring 2014 exam are provided in Table 26 (page 63). The average for all three categories was higher 
for students who earned the advanced standard than it was for those who met the phase-in 1, 
satisfactory standard. Students who achieved the advanced standard logged an average of 22 TTM 
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lessons completed, 11 passed, and four target TTM lessons passed while those who met the 
satisfactory standard completed an average of 18 TTM lessons, passed an average of seven, and 
passed an average of two target TTM lessons. 


Based on the number of TTM lessons they completed in 2013-2014, how did HISD students who 
attended 2013-2014 priority and focus schools and who met the spring 2013 STAAR mathematics 
phase-in 1, satisfactory standard perform on the spring 2014 STAAR mathematics assessment? 


e §=©A total of 5,955 students (47 percent of students in 2013-2014 priority and focus schools who took both 
the spring 2013 and 2014 STAAR mathematics exams) met the phase-in 1, satisfactory standard on 
the spring 2013 STAAR mathematics assessment. Fifty-two (52) percent of these students used TTM 
and 48 percent did not. Spring 2014 STAAR mathematics results for these students are shown in 
Figure 28 and in Table 27 (page 64). More than 80 percent of the students, TTM users and non-TTM 
users, who met the 2013 phase-in 1, satisfactory standard also met the standard on the spring 2014 
assessment. A larger percentage of TTM users than non-TTM users achieved both the satisfactory 
and the advanced standard on the same test. 


Figure 28. Spring 2014 STAAR mathematics performance of students who attended a priority or 
focus school, took both the spring 2013 and 2014 STAAR mathematics assessments, and 
met the phase-in 1, satisfactory standard on the spring 2013 exam 


100 5 

80 + 
& 

S 60 - 
¢ 
oa 
= 
[0 

o 40 4 

20 - 

9 
O 4 
Did Not Meet Standard Met Standard Met Advanced Standard 
&TTM Users &Non-TTM Users 
Note: Percentages may not equal 100 due to rounding. 


Sources: 2013 and 2014 STAAR mathematics files, TEA priority and focus school 
identification files, and TTM files 
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e = Illustrated in Figure 29 and shown in Table 28 (page 64), the majority of TTM users who met the phase- 
in 1, satisfactory standard on the spring 2013 STAAR mathematics assessment completed 1-14 TTM 
lessons, passed 1-14 TTM lessons, and also passed 1-14 target TTM lessons. Sixteen (16) percent 
completed 30 or more TTM lessons, as recommended by the TTM company, five percent passed 30 
or more TTM lessons, and one percent passed thirty or more target TTM lessons. 


Figure 29. Percentage of TTM users who attended a 2013-2014 priority or focus school and met 
the 2013 STAAR mathematics phase-in 1, satisfactory standard, by number of TTM lessons 
completed, number of TTM lessons passed, and number of 
target TTM lessons passed, 2013-2014 
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Sources: 2013 and 2014 STAAR mathematics files, TEA priority and focus school 
identification files, and TTM files 
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e Shown in Figure 30 and in Table 28 (page 64) the percentage of TTM users who successfully met the 
phase-in 1, satisfactory standard on the spring 2014 STAAR mathematics test after earning the 
satisfactory standard on the spring 2013 assessment, rose steadily from 43 percent of students who 
completed zero TTM lessons to 91 percent of students who completed 30 or more lessons. For 
comparison, shown in Figure 30 and in Table 27 (page 64), 82 percent of non-TTM users met the 
phase-in 1 satisfactory standard on both the spring 2013 and 2014 STAAR mathematics exams, slightly 
higher than the percentage recorded for TTM users who completed 1-14 TTM lessons. The percentage 
of TTM users who earned the 2014 STAAR mathematics advanced standard was larger than the 
percentage of non-TTM users for each category of number of lessons completed. 


Figure 30. Percentage of students who attended a 2013-2014 priority or focus schools and 
achieved satisfactory and advanced ratings on the spring 2014 STAAR mathematics assessment 
after meeting the satisfactory standard on the spring 2013 STAAR mathematics test, 
by number of TTM lessons completed and compared with students 
in priority and focus schools who did not enroll in TTM 
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Sources: 2013 and 2014 STAAR mathematics files, TEA priority and focus school 
identification files, and TTM files 


e Regression analyses examining the usefulness of the number of TTM lessons completed, number of 
TTM lessons passed, and number of target TTM lessons completed in predicting spring 2014 STAAR 
mathematics scale scores of students in priority and focus schools who met the phase-in 1, satisfactory 
standard on the spring 2013 STAAR mathematics assessment are shown in Table 29 (page 65). All 
of the relationships were significant, with higher numbers of lessons being associated with higher scale 
scores. The effect size of the relationship between number of TTM lessons passed and spring 2014 
STAAR mathematics scale scores and of the relationship between number of target TTM lessons and 
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spring STAAR mathematics 2014 STAAR mathematics scale scores were small, and the effect size of 
the relationship between the number of TTM lessons completed and spring 2014 scale scores was very 
small. 


e Asseen in Table 30 (page 65), students in priority and focus schools who met the satisfactory standard 
on the spring 2013 STAAR mathematics assessment and those who were also successful on the spring 
2014 STAAR mathematics exam had lower average numbers of lessons in every category than did 
students in the same group who earned the advanced standard on the 2014 exam. The ranges and 
standard deviations for all these averages were large. 


Based on the number of TTM lessons they completed in 2013-2014, how did students who attended 
priority and focus schools and who met the 2013 STAAR mathematics advanced standard perform 
on the spring 2014 STAAR mathematics assessment? 


e ~= Virtually all, all but two (0.6 percent), of the students at priority and focus schools who took both the 
spring 2013 and 2014 STAAR mathematics assessments and earned the advanced standard on the 
spring 2013 assessment, met the phase-in 1, satisfactory standard on the spring 2014 exam, and 56 
pecent of them also achieved the advanced standard. As seen in Figure 31 and detailed in Table 31 
(page 66), a larger percentage of TTM users earned the advanced standard on the spring 2014 STAAR 
mathematics assessment than did non-TTM users. 


Figure 31. Spring 2014 STAAR mathematics performance of students who attended a priority or 
focus school, took both the spring 2013 and 2014 STAAR mathematics assessments, and 
met the advanced standard on the spring 2013 exam 
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Sources: 2013 and 2014 STAAR mathematics files, TEA priority and focus school 
identification files, and TTM files 


HISD Research and Accountability. 36 


e The numbers of lessons completed and passed by TTM users who attended 2013-2014 priority and 
focus schools and who earned the advanced standard on the spring 2013 STAAR mathematics exam 
are shown in Figure 32. All of these students completed at least one TTM lesson and a higher 
percentage of these students were represented in each of the categories for 30 or more lessons than 
were TTM users in priority or focus schools who did not achieve the 2013 advanced standard (for 
comparison, see Figure 26, page 31, and Figure 29, page 34). Twenty-seven (27) percent completed 
30 or more TTM lessons, 16 percent passed 30 or more lessons, and six percent passed 30 or more 
target TTM lessons in 2013-2014. 


Figure 32. Percentage of TTM users who attended a 2013-2014 priority or focus school and met 
the 2013 STAAR mathematics advanced standard, by number of TTM lessons completed, 
number of TTM lessons passed, and number of target TTM lessons passed, 2013-2014 
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Sources: 2013 and 2014 STAAR mathematics files, TEA priority and focus school 
identification files, and TTM files 
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The spring 2014 STAAR mathematics results based on the number TTM lessons completed in 2013-— 
2014 for students at priority and focus schools who earned the 2013 STAAR mathematics advanced 
standard are presented in Figure 33 and are detailed in Table 32 (page 66). Virtually all of the students 
successfully met the phase-in 1, satisfactory standard on the spring 2014 STAAR mathematics 
assessment. The percentage of students earning the advanced standard increased as the number of 
completed TTM lessons increased, and the percentage of students who achieved the advanced 
standard was higher for each category of TTM users than it was for non-TTM users (Figure 33 and 
Table 31, page 66). 


Figure 33. Percentage of students who attended a 2013-2014 priority or focus schools and 
achieved satisfactory and advanced ratings on the spring 2014 STAAR mathematics 
assessment after meeting the advanced standard on the spring 2013 STAAR 
mathematics test, by number of TTM lessons completed and compared 
with students in priority and focus schools who did not enroll in TTM 
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Sources: 2013 and 2014 STAAR mathematics files, TEA priority and focus school 
identification files, and TTM files 


Regression analyses using the number of TTM lessons completed, the number of TTM lessons passed, 
and the number of target TTM lessons passed to predict spring 2014 STAAR mathematics scale scores 
of students in 2013-2014 priority and focus schools who achieved the advanced standard on the 2013 
STAAR mathematics assessment are reported in Table 33 (page 67). None of the relationships were 
significant, indicating that the numbers of TTM lessons were not predictors of spring 2014 STAAR 
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mathematics scale scores for students in priority and focus schools who achieved the advanced 
standard on the spring 2013 STAAR mathematics assessment. 


e Students at priority and focus schools who earned the advanced standard on both the spring 2013 and 
2014 STAAR mathematics exams completed an average of 26 TTM lessons, passed an average of 20 
lessons, and passed an average of 13 target TTM lessons. As shown in Table 34 (page 67), averages 
were even higher for the students who met the lower, satisfactory, standard on the test, with the 
exception of the average number of target TTM lessons passed, which was slightly lower. 


How many HISD students used TTM in 2013-2014 and took the June 2014 retest of the 2014 STAAR 
mathematics assessment, and how did they compare with their peers in the district? 


e Students in grades five and eight who were unsuccessful on the spring 2014 STAAR mathematics 
assessment were offered the opportunity to retake the test in June 2014. Of the students who had 
taken both the spring 2013 and 2014 STAAR mathematics exams, 2,789 students in grades five and 
eight who had been unsuccessful on the spring 2014 assessment retook the test. Demographic 
information for these students, including numbers that allow calculation of the results in Figures 34-37 
(pages 39-42) can be found in Table 35 (page 68). Fifty-three (53) percent of these students, 1,478 
students, used TTM in 2013-2014 and 1,311, 47 percent, did not. Illustrated in Figure 34, 62 percent 
of retested students in priority schools and 52 percent of retested students in focus schools used TTM. 
These percentages largely parallel those of the group of all HISD students who took both the spring 
2013 and 2014 STAAR mathematics tests (see Figure 3, page 8 for comparison). 


Figure 34, Percentage of TTM users and non-TTM users among students who took both 
the spring 2013 and 2014 STAAR mathematics assessments and retook the 2014 STAAR 
mathematics exam in June 2014, by attendance at 2013-2014 priority and focus schools 
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e Shown in Figure 35, 71 percent of the fifth graders who took both the spring 2013 and 2014 spring 
STAAR mathematics tests and retested in June 2014 used TTM and 29 percent did not, while in eighth- 
grade, the percentages were reversed: 29 percent used TTM and 71 percent did not. This finding 
echoes the results depicted in Figure 4 (page 9) and Figure 20 (page 25), with majorities of students in 
grades three through five using TTM and majorities of students in grades six through eight not using 
the program. 


Figure 35. Percentage of TTM users and non-TTM users among students who took both the spring 
2013 and 2014 STAAR mathematics assessments and retook the 2014 STAAR 
mathematics exam in June 2014, by 2013-2014 grade level 
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e By gender, 53 percent of both females and males who retook the 2014 STAAR mathematics 
assessment in June were TTM users, shown in Figure 36. In comparison with the whole sample of 
retesters, a larger percentage of African American, white, and other students, and a smaller percentage 


of Asian/Pacific Islander and Hispanic students used TTM than did in the larger population. 


Figure 36. Percentage of TTM users and non-TTM users among students who took both the 
spring 2013 and 2014 STAAR mathematics assessments and retook the 2014 STAAR 


mathematics exam in June 2014, by gender and race/ethnicity 
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e = Illustrated in Figure 37, a higher proportion of retested students with economic disadvantage other than 
free and reduced lunch and a lower proportion of students with reduced lunch and no economic 
disadvantage used TTM in 2013—2014 compared with students in the whole sample of students who 
retook the 2014 STAAR mathematics assessment in June 2014. _ Fifty-one (51) percent of LEP 
retesters used TTM while among non-LEP students, a higher percentage used TTM than did not. 


Figure 37. Percentage of TTM users and non-TTM users among students who took both the spring 
2013 and 2014 STAAR mathematics assessments and retook the spring 2014 STAAR 
mathematics exam in June, by economic disadvantage and LEP status 
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e More detail on demographics of students who took both the spring 2013 and 2014 STAAR mathematics 
assessments and retook the 2014 exam in June can be found in Table 35 (page 68). 
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e =Spring 2013 STAAR mathematics results for students who retested in June 2014 are shown in Figure 
38 and in Table 36 (page 69). Students who were not successful on the spring 2014 STAAR 
mathematics assessment were largely also unsuccessful the year before, on the spring 2013 exam. 
Nine percent of TTM users met the phase-in 1, satisfactory standard and 14 percent of non-TTM users 
met the satisfactory standard on the 2013 test. Only one TTM user and one non-TTM user achieved 
the advanced standard. 


Figure 38. Spring 2013 STAAR mathematics performance of students who took both the 
spring 2013 and 2014 STAAR mathematics assessments and retook the 2014 STAAR 
mathematics exam in June 2014, by TTM useage 
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e The June 2014, retest, results for students who took both the spring 2013 and 2014 STAAR 
mathematics exams and did not meet the phase-in 1 standard on the spring 2014 assessment are 
shown in Figure 39 and Table 37 (page 69). The majority of students were again unsuccessful in 
attaining the satisfactory standard, however, 26 percent of TTM users and 23 percent of non-TTM users 
were successful. No retesters in the group earned the advanced standard on the June 2014 retest. 


Figure 39. June 2014, retest, STAAR mathematics performance of students who took both the 
spring 2013 and 2014 STAAR mathematics assessments and retook the 2014 STAAR 
mathematics exam in June 2014, by TTM useage 
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Sources: 2013 and 2014 STAAR mathematics files and TTM files 


e Apropensity score nearest neighbor matching anaysis was conducted to compare the June 2014 retest 
scale scores of TTM users with the scale scores earned by non-TTM users, all of whom had also taken 
the spring 2013 and 2014 spring STAAR mathematics assessments. Students were matched on their 
spring 2013 STAAR mathematics scale scores, attendance at a priority or focus school, grade level, 
gender, economic disadvantage status, LEP status, and race/ethnicity. All 2,789 students were 
included in the analysis. The results, shown in Table 38 (page 70), showed no significant differences 
in scale scores between TTM users and non-TTM users after students were matched. 


Discussion 


The Think Through Math program was relatively widely used in HISD in 2013-2014. Unique users who 
could be identified numbered 51,863 and were in every grade level. In grades three through eight, the 
grades levels for which TTM was funded by the Texas Education Agency, 53 percent of the students who 
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took both the spring 2013 and spring 2014 STAAR mathematics assessments used TTM. The program 
had even higher usage rates in 2013-2014 priority schools; 66 percent of students in priority schools who 
took both the spring 2013 and 2014 STAAR mathematics assessments used TTM. The program was also 
well used by HISD students who did not meet the spring 2013 STAAR mathematics phase-in 1, satisfactory 
standard, as 56 percent of these students were TTM users. 


With the exception of students who retested on the STAAR mathematics assessment in June 2014, 
propensity score matching results indicated that students who used TTM earned higher average scale 
scores on the spring 2014 STAAR mathematics assessment than did similar students who did not use TTM. 


Regression analyses were used to establish relationships between spring 2014 STAAR mathematics scale 
scores for groups of students based on spring 2013 STAAR mathematics results and three categories of 
TTM lessons: number of TTM lessons completed, number of TTM lessons passed, and number of target 
TTM lessons completed. For the majority of groups, the numbers of TTM lessons were significant predictors 
of 2014 STAAR mathematics scale scores, with higher numbers of lessons associated with higher scale 
scores. However, the category of TTM lessons matters. The strongest relationship was for the number of 
target TTM lessons passed by HISD students who met the 2013 STAAR mathematics phase-in 1, 
satisfactory standard, for which the significant positive relationship had a medium effect size. The higher 
the number of target TTM lessons an HISD student who met the satisfactory standard on the spring 2013 
STAAR mathematics exam passed, the higher the student’s scale score on the spring 2014 STAAR 
mathematics was likely to be. Weak relationships were found for students at priority and focus schools who 
earned the advanced standard on the spring 2013 STAAR mathematics exam. None of the categories for 
TTM lessons, those completed, those passed, or target TTM lessons passed, was a significant predictor of 
spring 2014 STAAR mathematics scale scores. However, a similar weak relationship was found for the 
number of TTM lessons completed by TTM users who did not meet the satisfactory standard on the 2013 
STAAR mathematics assessment. More strikingly, the number of TTM lessons completed by students who 
did not meet the satisfactory standard on the 2013 STAAR mathematics exam and who attended a priority 
or focus school did predict spring 2014 STAAR mathematics scale scores, but the relationship was 
negative. The more TTM lessons these students completed, the lower the 2014 STAAR mathematics scale 
score was likely to be. For these same students, the number of TTM lessons passed and the number of 
target TTM lessons passed were both significant predictors of 2014 STAAR mathematics scale scores, and 
the relationships were positive. The more TTM lessons passed by a student in a priority or focus school 
who did not meet the 2013 STAAR mathematics satisfactory standard, and the more target TTM lessons 
the student passed, the higher the 2014 STAAR mathematics scale score was likely to be. The number 
of TTM lessons passed and the number of target TTM lessons passed were more consistent predictors of 
higher 2014 STAAR mathematics scale scores than the number of TTM lessons completed was. 


The optimal number of TTM lessons (or target TTM lessons) for a student to complete or pass in order to 
increase achievement on the STAAR mathematics assessment is not clear from this analysis. The TTM 
company recommends that students “think 30,” i.e., complete—and preferably pass—30 TTM lessons to 
use the program with fidelity. Only 16 percent of TTM users in this analysis completed 30 or more lessons, 
while six percent passed 30 or more TTM lessons and two percent passed 30 or more target TTM lessons. 
The average numbers of lessons completed and passed by students who successfully met the 2014 phase- 
in 1 standard on the spring 2014 STAAR mathematics exam were all below 28, and the standard deviations 
for the averages were very large, indicating great variation in the numbers of lessons associated with 
succeeding on the 2014 STAAR mathematics exam. Also notable, the range of courses completed by 
successful students began with 0 to 4, indicating relatively minimal contact with TTM for some successful 
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students. In addition, with one exception, effect sizes for relationships between the number of lessons 
completed or passed and 2014 scale scores were small. These findings all suggest that the number of 
TTM lessons associated with success on the STAAR mathematics assessment varies greatly. 


The results of this study show that TTM lessons were largely helpful in supporting the STAAR mathematics 
achievement of HISD students who used them. The wide dispersal of the average number of lessons 
associated with success on the STAAR and the results for some groups of students, specifically students 
who did not achieve the passing standard on the previous year’s STAAR mathematics assessment and 
students at priority and focus schools who achieved the advanced standard on the same test, indicate that 
some students did not benefit as much as others did. Though the potential value of the program is clear, it 
is also clear that select groups of students may need additional attention to achieve higher STAAR 
mathematics scale scores. TTM can be a useful tool in supporting student achievement on the STAAR 
mathematics exam, but is likely best used as recommended by the program creators, with significant 
teacher and monitor oversight and as a supplement in conjunction with other successful methods for diverse 
learners. 
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Appendix A 


Data Limitations 


For 2013-2014, TTM files were organized based on user name. They contained a single identification 
number, the Texas Student Data System (TSDS) ten-digit unique student identification number, as required 
by the state of Texas. TSDS identification numbers were provided by school personnel over the course of 
the year. Because the TSDS was relatively new and school personnel were accustomed to providing 
seven-digit local student identification numbers, many personnel filled in the ten-digit numbers in ways that 
made the data accessible to the school but which did not correlate with TSDS identification numbers. Asa 
result, more than half of the 55,585 TSDS identification numbers for students who used TTM that were 
provided by the TTM company did not match the TSDS numbers provided by the state. Through matching 
the original numbers with data available within the district, including local student identification numbers, 
school numbers, student names and schools, and so on, TSDS numbers were found for 52,010 unique 
TTM student users. Data for students without TSDS numbers, including homeschooled students, were not 
included in the analyses of TTM users. Further, of the 52,010 unique TTM student users, 51,863 could be 
matched to 2014 PEIMS Fall Resubmission data for demographic information and were included in the 
analyses. As a result, for comparisons of performance, some students who used TTM but were not 
identifiable were included in the group of students who did not use TTM. 


Students could create multiple accounts, and many did, for working from home, after changing schools, for 
working with different teachers in the same school, and so on. Eight students had four accounts, 48 had 
three, and 1,259 had two, for a total of 1,315 students with multiple accounts. For analyses, the numbers 
of TTM lessons that the 51,863 unique students completed were summed and associated with the first 
school assigned by TTM. The majority of students with multiple accounts had all accounts linked to a single 
school. For the remainder, the majority were linked with schools that were in the same category of 
priority/focus school or not. However, the method resulted in some lessons being credited to schools at 
which they were not completed. 


TTM data collection methods included resetting all values when identification numbers were changed. As 
a result, for the second draw of data, for TTM work done by June 23, 2014, TTM data for 180 of 1,478 
students who used TTM and retested in June was missing and the remaining accounts contained similar 
kinds of errors in identification numbers that were found in the first file. A large percentage of the data from 
the second draw was not reliably accessible. As a result, number of lessons completed, number of lessons 
passed, and number of target lessons passed were not reported for students who retested in June 2014. 
Test results for TTM students were reported and were compared with those of retested students who did 
not use TTM, but the numbers of lessons associated with TTM students were not available to be included 
in the analysis. 
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Appendix B 


Table 1. Demographic Characteristics and TTM Achievement of Students Who Used TTM, 


7A0 ey A 
Mean # Mean # Mean # Target 
Lessons Lessons Lessons 
N % Completed Passed Completed 
School Attended 
Priority School 4,880 9.4 17.2 5.7 1.9 
Focus School 10,021 19.3 17.3 6.0 2.2 
Not a Prioritized School 36,962 71.3 17.8 9.2 4.5 
Grade Level 
PreKindergarten 5 <0.1 87.8 5.6 0.6 
Kindergarten 6 <0.1 3.8 2.3 0.5 
Grade 1 6 <0.1 9.5 3.8 1.0 
Grade 2 101 0.2 17.7 8.0 4.2 
Grade 3 11,944 23.0 18.3 75 3.9 
Grade 4 11,781 22.7 20.5 9.5 4.6 
Grade 5 11,407 22.0 18.4 9.0 3.7 
Grade 6 5,115 9.9 15.7 8.2 4.1 
Grade 7 5,098 9.8 15.2 7.5 3.0 
Grade 8 3,676 7.1 13.8 7.1 3.4 
Grade 9 2,143 4.1 13.5 6.8 2.7 
Grade 10 352 0.7 9.5 4.3 1.5 
Grade 11 122 0.2 12.9 5.0 1.7 
Grade 12 107 0.2 16.1 5.4 1.3 
Gender 
Female 25,345 48.9 15.5 75 3.4 
Male 26,518 51.1 19.7 9.0 4.2 
Race/Ethnicity 
African American 13,652 26.3 16.6 6.7 2.6 
American Indian 109 0.2 20.8 10.0 4.7 
Asian/Pacific Islander 1,609 3.1 32.1 22.1 14.9 
Hispanic 32,448 62.6 17.2 7.6 3.2 
White 3,610 7.0 19.4 13.0 8.1 
Two or more 435 0.8 19.7 12.4 7.6 
Economic Disadvantage 
Free Lunch 17,203 33.2 18.1 7.8 3.3 
Reduced Lunch 3,726 7.2 18.6 9.4 4.5 
Other Economic Disadvantage 22,328 43.1 16.3 6.5 2.5 
No Economic Disadvantage 8,606 16.6 20.0 13.1 8.0 
Limited English Proficiency (LEP) 
LEP 20,626 39.8 18.9 7.9 3.3 
Not LEP 31,237 60.2 16.8 8.5 4.2 
Total 51,863 100.0 17.7 8.2 3.8 
Note: Percentages may not total 100 due to rounding 


Sources: PEIMS Fall Resubmission and TTM files 
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Table 2. Demographic Characteristics of HISD Students Who Took the Spring 2013 and 


2014 STAAR Mathematics Assessments and Were Used in Analyses, by TTM 


Usage 
TTM Users Non-TTM Users Total 
N % N % N 
School Attended 
Priority School 1,388 4.7 710 2.8 2,098 
Focus School 5,631 19.2 5,063 19.7 10,694 
Not a Prioritized School 22,340 76.1 19,945 77.6 42,285 
Grade Level 
Grade 3 382 1.3 154 0.6 536 
Grade 4 9,661 32.9 3,758 14.6 13,419 
Grade 5 9,403 32.0 3,373 13.1 12,776 
Grade 6 3,883 13.2 6,440 25.0 10,323 
Grade 7 3,846 13.1 6,497 25.3 10,343 
Grade 8 2,184 7.4 5,496 21.4 7,680 
Gender 
Female 14,479 49.3 12,928 50.3 27,407 
Male 14,880 50.7 12,790 49.7 27,670 
Race/Ethnicity 
African American 6,561 22.3 5,568 21.7 12,129 
American Indian 62 0.2 37 0.1 99 
Asian/Pacific Islander 925 3.2 887 3.4 1,812 
Hispanic 19,475 66.3 16,965 66.0 36,440 
White 2,110 2 2,086 8.1 4,196 
Two or more 226 0.8 175 0.7 401 
Economic Disadvantage 
Free Lunch 8,997 30.6 8,025 31.2 17,022 
Reduced Lunch 2,220 7.6 1,902 7.4 4,122 
Other Economic Disadvantage 13,167 44.8 10,846 42.2 24,013 
No Economic Disadvantage 4,975 16.9 4,945 19.2 9,920 
Limited English Proficiency (LEP) 
LEP 12,596 42.9 10,028 39.0 22,624 
Not LEP 16,763 57.1 15,690 61.0 32,453 
Total 29,359 100.0 25,718 100.0 55,077 


Notes: Students in grade three are those who repeated the grade; they were enrolled in 
grade three for both the spring 2013 and 2014 STAAR mathematics 
assessments. 

Percentages may not total 100 due to rounding. 

Source: PEIMS Fall Resubmission and TTM files 
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Table 3. Spring 2013 STAAR Mathematics Results for HISD Students Who 


solo) qu =10) 40] 


the Spring 2013 and 2014 STAAR Mathematics 
Assessments, by Use of TTM 


Did Not Meet Achieved the 
the 2013 Met the 2013 2013 
Phase-In 1 Phase-In 1 Advanced 
Total Standard Standard Standard° 
N N % N N N % 
TTM Users 29,359 | 10,227 34.8 19,132 65.2 | 4,262 14.5 
Non-TTM Users 25,718 8,095 31.5 17,623 68.5 3,619 14.1 
Total 55,077 18,322 33.3 36,755 66.7 7,881 14.3 
Notes: °Students are also included in the number “Met the 2013 Phase-In 1 
Standard.” 
Remaining percentages may not equal 100 due to rounding. 
Sources: 2013 STAAR, 2014 STAAR, 10-8-13 SharePoint listing of 2013-2014 


priority and focus schools, and TTM files 


Table 4. Spring 2014 STAAR Mathematics Results for HISD Students Who Also 


Took the Spring 2013 STAAR Mathematics Assessment, by Use of TTM 


Did Not Meet Achieved the 
the 2014 Met the 2014 2014 
Phase-In 1 Phase-In 1 Advanced 
Total Standard Standard Standard° 
N N % N N N % 
TTM Users 29,359 8,394 28.6 20,965 71.4 5,378 18.3 
Non-TTM Users 25,718 7,320 28.5 18,398 71.5 | 3,824 14.9 
Total 55,077 | 15,714 28.5 39,363 71.5 | 9,202 16.7 
Notes: °Students are also included in the number “Met the 2014 Phase-In 1 
Standard.” 
Remaining percentages may not equal 100 due to rounding. 
Sources: 2013 STAAR, 2014 STAAR, and TTM files 


Table 5. Propensity Score Matching Results for All HISD Students Who Took Both the 


Spring 2013 and 2014 STAAR Mathematics Assessments, by TTM Usage 


Mean 2014 STAAR 
Mathematics Scale Score 
TTM 
Users Non-TTM Users 
(N=29,359) (N=25,718) Difference S.E. t 
Before Matching 1,589.3 1,612.9 -23.7 1.3 -18.69* 
Matched 1,589.3 1,577.6 11.7 3.4 3.43* 


Note: * indicates p<.001. 
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Table 6. Spring 2014 STAAR Mathematics Results for HISD Students Who Did 


Not Meet the Level 2, Satisfactory, Phase-In 1 Standard on the Spring 
2013 STAAR Mathematics Assessment, by Use of TTM 


Did Not Meet Achieved the 
the 2014 Met the 2014 2014 

Phase-In 1 Phase-In 1 Advanced 

Total Standard Standard Standard° 

N N % N N N % 

TTM Users 10,227 6,429 62.9 3,798 37.1 73 0.7 

Non-TTM Users 8,095 5,138 63.5 2,957 36.5 40 O05 

Total 18,322 | 11,567 63.1 6,755 36.9 1130.6 
Notes: °Students are also included in the number “Met the 2014 Phase-In 1 


Standard.” 
Remaining percentages may not equal 100 due to rounding. 

Sources: 2013 STAAR, 2014 STAAR, 10-8-13 SharePoint listing of 2013-2014 
priority and focus schools, and TTM files 


Table 7. Spring 2014 STAAR Mathematics Results for TTM Users Who Did Not Meet the 
Level 2, Satisfactory, Phase-In 1 Standard on the Spring 2013 STAAR 


Mathematics Assessment, by Number of TTM Lessons Completed, Number of 
TTM Lessons Passed, and Number of TTM Target Lessons Passed 


Did Not Meet the Met the 2014 Achieved the 
2014 Phase-In 1 Phase-In 1 2014 Advanced 
Total Standard Standard Standard° 
N N % N N N % 

TTM Lessons Completed 

0 29 18 62.1 11 37.9 0 0.0 

1-14 6,276 4,049 64.5 | 2,227 35.5 31 0.5 

15-29 2,536 1,549 61.1 987 38.9 24 0.9 

30 or more 1,386 813 58.7 573 41.3 18 1.3 
TTM Lessons Passed 

0 983 805 81.9 178 18.1 2 0.2 

1-14 8,606 5,391 62.6 3,215 37.4 50 0.6 

15-29 482 183 38.0 299 62.0 13 2.7 

30 or more 156 50 32.1 106 67.9 8 5.1 
TTM Target Lessons Passed 

0 6,192 4,504 72.7 | 1,688 27.3 18 0.3 

1-19 3,983 1,913 48.0 2,070 52.0 50 1.3 

20-29 33 9 27.3 24 72.7 4 12.1 

30 or more 19 3 15.8 16 84.2 1 5.3 
Total 10,227 6,429 62.9 | 3,798 37.1 73 0.7 

Notes: °Students are also included in the number “Met the 2014 Phase-In 1 Standard.” 


Percentages may not total 100 due to rounding. 
Sources: 2013 STAAR, 2014 STAAR, and TTM files 
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Table 8. Regression Analyses of Relationships between 2014 STAAR Mathematics Scale Scores 
and the Number of TTM Lessons Completed, TTM Lessons Passed, and TTM Target 


Lessons Passed by TTM Users Who Took Both the Spring 2013 and 2014 STAAR 
Mathematics Assessments and Did Not Meet the Level 2, Satisfactory, Phase-In 1 
Standard on the Spring 2013 STAAR Mathematics Assessment 


Independent Variable Dependent Variable t R? 
Number of TTM Lessons 2014 STAAR Mathematics 
-.01 t (10,225) = -1.01 31 NA 
Completed Scale Score 
Number of TTM Lessons 2014 STAAR Mathematics 
.20 t (10,225) = 20.57 < .001 .04 
Passed Scale Score 
Number of TTM Target 2014 STAAR Mathematics 
16 t (10,225) = 16.19 < .001 .03 
Lessons Passed Scale Score 


Note: Effect size conventions for R? are: .01 is small, .09 is medium, and .24 is large. 


Table 9. Number of TTM Lessons Recorded for TTM Users Who Did Not Meet the 


Spring 2013 STAAR Level 2, Satisfactory, Phase-In 1 Standard, by Spring 
2014 STAAR Mathematics Performance, 2013-2014 


Number of Lessons 
Standard 
Range Mean Deviation 
Met 2014 Phase-In 1 Standard (N=3,798) 
TTM Lessons Completed 0-276 17.5 20.9 
TTM Lessons Passed 0-136 7.1 9.1 
TTM Target Lessons Passed 0-79 2.0 4.4 
Met 2014 Advanced Standard (N=73) 
TTM Lessons Completed 1-127 23.8 23.1 
TTM Lessons Passed 0-74 12.0 12.9 
TTM Target Lessons Passed 0-46 5.0 7.6 


Sources: 2013 STAAR, 2014 STAAR, and TTM files 
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Table 10. Spring 2014 STAAR Mathematics Results for HISD Students Who 


Met the Level 2, Satisfactory, Phase-In 1 Standard on the Spring 
2013 STAAR Mathematics Assessment, by Use of TTM 


Did Not Meet Achieved the 
the 2014 Met the 2014 2014 

Phase-In 1 Phase-In 1 Advanced 

Total Standard Standard Standard° 

N N % N N N % 

TTM Users 19,132 1,965 10.3 17,167 89.7 | 5,305 27.7 
Non-TTM Users 17,623 2,182 12.4 15,441 87.6 | 3,784 21.5 
Total 36,755 4,147 11.3 32,608 88.7 | 9,089 24.7 
Notes: °Students are also included in the number “Met the 2014 Phase-In 1 


Standard.” 
Remaining percentages may not equal 100 due to rounding. 

Sources: 2013 STAAR, 2014 STAAR, 10-8-13 SharePoint listing of 2013-2014 
priority and focus schools, and TTM files 


Table 11. Spring 2014 STAAR Mathematics Results for TTM Users Who Met the Level 2, 
Satisfactory, Phase-In 1 Standard on the Spring 2013 STAAR Mathematics 


Assessment, by Number of TTM Lessons Completed, Number of TTM Lessons 
Passed, and Number of TTM Target Lessons Passed 


Did Not Meet the Met the 2014 Achieved the 
2014 Phase-In 1 Phase-In 1 2014 Advanced 
Total Standard Standard Standard° 
N N % N N N % 

TTM Lessons Completed 

0 41 8 19.5 33 80.5 15 36.6 

1-14 11,472 1,347 11.7 10,125 88.3 2,742 23.9 

15-29 4,346 404 9.3 3,942 90.7 1,242 28.6 

30 or more 3,273 206 6.3 3,067 93.7 1,306 39.9 
TTM Lessons Passed 

0 531 180 33.9 351 66.1 41 7.7 

1-14 14,347 1,666 11.6 12,681 88.4 3,200 22.3 

15-29 2,702 92 3.4 2,610 96.6 1,163 43.0 

30 or more 1,552 27 1.7 1,525 98.3 901 58.1 
TTM Target Lessons Passed 

0 4,863 1,183 24.3 3,680 75.7 398 8.2 

1-19 12,796 772 6.0 12,024 94.0 3,873 30.3 

20-29 827 7 0.8 820 99.2 554 67.0 

30 or more 646 3 0.5 643 99.5 480 74.3 
Total 19,132 1,965 10.3 17,167 89.7 5,305 27.7 


Notes: °Students are also included in the number “Met the 2014 Phase-In 1 Standard.” 
Percentages may not total 100 due to rounding. 
Sources: 2013 STAAR, 2014 STAAR, and TTM files 
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Table 12. Regression Analyses of Relationships between Spring 2014 STAAR Mathematics Scale 
Scores and the Number of TTM Lessons Completed, TTM Lessons Passed, and TTM 


Target Lessons Passed by TTM Users Who Took Both the Spring 2013 and 2014 STAAR 
Mathematics Assessments and Met the Level 2, Satisfactory, Phase-In 1 Standard on 


the Spring 2013 STAAR Mathematics Assessment 


Independent Variable Dependent Variable t R? 
Number of TTM Lessons 2014 STAAR Mathematics 
11 t (19,130) = 15.41 | <.001 | .01 
Completed Scale Score 
Number of TTM Lessons 2014 STAAR Mathematics 
.24 t (19,130) = 34.77 < .001 .06 
Passed Scale Score 
Number of TTM Target 2014 STAAR Mathematics 
34 t (19,130) = 50.51 | <.001 | .12 
Lessons Passed Scale Score 


Note: Effect size conventions for R? are: .01 is small, .09 is medium, and .24 is large. 


Table 13. Number of TTM Lessons Recorded for TTM Users Who Met the Spring 


2013 STAAR Mathematics Level 2, Satisfactory, Phase-In 1 Standard, 
by 2014 STAAR Mathematics Performance, 2013-2014 


Number of Lessons 
Standard 
Range Mean Deviation 
Met 2014 Phase-In 1 Standard (N=17,167) 
TTM Lessons Completed 0-1,110 18.8 26.1 
TTM Lessons Passed 0-1,050 12.0 19.0 
TTM Target Lessons Passed 0-510 6.3 12.3 
Met 2014 Advanced Standard (N=5,305) 
TTM Lessons Completed 1-453 23.2 29.4 
TTM Lessons Passed 0-310 17.7 22.8 
TTM Target Lessons Passed 0-261 11.6 16.5 


Sources: 2013 STAAR, 2014 STAAR, and TTM files 
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Table 14. Spring 2014 STAAR Mathematics Results for HISD Students Who 


Met the Level 3, Advanced Performance, on the Spring 2013 STAAR 
Mathematics Assessment, by Use of TTM 


Did Not Meet Achieved the 
the 2014 Met the 2014 2014 
Phase-In 1 Phase-In 1 Advanced 
Total Standard Standard Standard° 
N N % N N N % 
TTM Users 4,262 24 0.6 4,238 99.4 3,012 70.7 
Non-TTM Users 3,619 19 0.5 3,600 99.5 | 2,373 65.6 
Total 7,881 43 0.5 7,838 99.5 | 5,385 68.3 
Notes: °Students are also included in the number “Met the 2014 Phase-In 1 
Standard.” 


Remaining percentages may not equal 100 due to rounding. 
2013 STAAR, 2014 STAAR, 10-8-13 SharePoint listing of 2013-2014 priority and 
focus schools, and TTM files 


Sources: 


Table 15. Spring 2014 STAAR Mathematics Results for TTM Users Who Met the Level 3, 
Advanced Performance Standard, on the Spring 2013 STAAR Mathematics 


Assessment, by Number of TTM Lessons Completed, Number of TTM Lessons 
Passed, and Number of TTM Target Lessons Passed 


HISD Research and Accountability 


Did Not Meet the Met the 2014 Achieved the 
2014 Phase-In 1 Phase-In 1 2014 Advanced 
Total Standard Standard Standard° 
N N % N N N % 

TTM Lessons Completed 

0 x * * * * * x 

1-14 2,236 16 0.7 2,220 99.3 1,485 66.4 

15-29 999 4 0.4 995 99.6 730 73.1 

30 or more 1,023 4 0.4 1,019 99.6 793 77.5 
TTM Lessons Passed 

0 37 3 8.1 34 91.9 19 51.4 

1-14 2,570 17 0.7 2,553 99.3 1,675 65.2 

15-29 938 0.3 935 99.7 732 78.0 

30 or more 717 1 0.1 716 99.9 586 81.7 
TTM Target Lessons Passed 

0 338 15 4.4 323 95.6 133 39.3 

1-19 3,061 9 0.3 3,052 99.7 2,133 69.7 

20-29 444 0 0.0 444 100.0 380 85.6 

30 or more 419 0 0.0 419 100.0 366 87.4 
Total 4,262 24 0.6 4,238 99.4 3,012 70.7 

Notes: °Students are also included in the number “Met the 2014 Phase-In 1 Standard.” 


*Results are not provided for fewer than five students. 
°Students are also included in the number “Met the 2014 Phase-In 1 Standard.” 
Sources: 2013 STAAR, 2014 STAAR, and TTM files 
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Table 16. Regression Analyses of Relationships between 2014 STAAR Mathematics Scale Scores 
and the Number of TTM Lessons Completed, TTM Lessons Passed, and TTM Target 


Lessons Passed by TTM Users Who Took Both the Spring 2013 and 2014 STAAR 
Mathematics Assessments and Met the Level 3, Advanced Performance Standard on 


the Spring 2013 STAAR Mathematics Assessment 


Independent Variable Dependent Variable t R? 
Number of TTM Lessons 2014 STAAR Mathematics 
.08 t (4,260) = 5.01 < .001 01 
Completed Scale Score 
Number of TTM Lessons 2014 STAAR Mathematics 
14 t (4,260) = 8.87 <.001 | .02 
Passed Scale Score 
Number of TTM Target 2014 STAAR Mathematics 
.23 t (4,260) = 15.57 < .001 .05 
Lessons Passed Scale Score 


Note: Effect size conventions for R? are: .01 is small, .09 is medium, and .24 is large. 


Table 17. Number of TTM Lessons Recorded for TTM Users Who Met the Spring 


2013 STAAR Mathematics Level 3, Advanced Performance Standard, 
by 2014 STAAR Mathematics Performance, 2013-2014 


Number of Lessons 
Standard 
Range Mean Deviation 
Met 2014 Phase-In 1 Standard (N=4,238) 
TTM Lessons Completed 0-1,100 23.0 34.0 
TTM Lessons Passed 0-1,050 18.1 28.6 
TTM Target Lessons Passed 0-510 12.2 18.8 
Met 2014 Advanced Standard (N=3,012) 
TTM Lessons Completed 1-334 24.2 29.7 
TTM Lessons Passed 0-310 19.8 24.8 
TTM Target Lessons Passed 0-261 14.1 18.5 


Sources: 2013 STAAR, 2014 STAAR, and TTM files 
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Table 18. Demographic Characteristics and TTM Achievement of Students Who Attended a 


Priority or Focus School and Used TTM, 2013-2014 


Mean # 
Mean # Mean # Target 
Lessons Lessons Lessons 
N % Completed Passed Completed 
School Attended 
Priority School 4,880 32.7 17.2 5.7 1.9 
Focus School 10,021 67.3 17.3 6.0 2.2 
Grade Level 
PreKindergarten % % x ss * 
Kindergarten a * * z= * 
Grade 1 * % * = * 
Grade 2 11 0.1 5.5 3.1 1.5 
Grade 3 3,063 20.6 18.5 5.4 2.5 
Grade 4 2,969 19.9 21.6 7.9 3.2 
Grade 5 2,675 18.0 20.1 7.2 2.1 
Grade 6 1,878 12.6 12.7 4.0 1.2 
Grade 7 2,136 14.3 15.3 5.3 1.3 
Grade 8 1,351 9.1 15.6 5.0 1.4 
Grade 9 641 4.3 4.9 2.9 1.3 
Grade 10 106 0.7 3.5 1.7 0.4 
Grade 11 27 0.2 5.3 2.2 0.4 
Grade 12 39 0.3 15.0 6.3 1.6 
Gender 
Female 7,238 48.6 15.2 5.3 1.8 
Male 7,663 51.4 19.3 6.4 2.3 
Race/Ethnicity 
African American 6,010 40.3 16.8 5.4 1.8 
American Indian 34 0.2 17.4 5.6 1.4 
Asian/Pacific Islander 124 0.8 51.1 13.5 5.0 
Hispanic 8,388 56.3 16.9 6.0 2.2 
White 288 1.9 22.4 al 2.9 
Two or more 57 0.4 11.8 6.2 3.2 
Economic Disadvantage 
Free Lunch 5,248 35.2 19.4 6.5 2.4 
Reduced Lunch 847 5.7 17.9 6.6 2.5 
Other Economic Disadvantage 7,772 52.2 15.9 5.3 1.8 
No Economic Disadvantage 1,034 6.9 16.0 6.5 2.7 
Limited English Proficiency (LEP) 
LEP 5,452 36.6 20.9 6.7 2.4 
Not LEP 9,449 63.4 15.2 5.4 1.9 
Total 14,901 100.0 17.3 5.9 2.1 


Notes: *Results are not provided for fewer than five students. 
Percentages may not total 100 due to rounding. 
Sources: PEIMS Fall Resubmission and TTM files 
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Table 19. Demographic Characteristics of HISD Students Who Attended a Priority or Focus 


School, Took the Spring 2013 and 2014 STAAR Mathematics Assessments, and 
Were Used in Analyses, by TTM Usage 


TTM Users Non-TTM Users Total 
N % N % N 
School Attended 
Priority School 1,388 19.8 710 12.3 2,098 
Focus School 5,631 80.2 5,063 87.7 10,694 
Grade Level 
Grade 3 116 1.7 18 0.3 134 
Grade 4 2,145 30.6 378 6.5 2,523 
Grade 5 1,955 27.9 430 7.4 2,385 
Grade 6 721 10.3 1,238 21.4 1,959 
Grade 7 1,357 19.3 1,851 32.1 3,208 
Grade 8 725 10.3 1,858 32.2 2,583 
Gender 
Female 3,439 49.0 2,865 49.6 6,304 
Male 3,580 51.0 2,908 50.4 6,488 
Race/Ethnicity 
African American 2,501 35.6 1,825 31.6 4,326 
American Indian 14 0.2 8 0.1 22 
Asian/Pacific Islander AT 0.7 70 1.2 117 
Hispanic 4,326 61.6 3,748 64.9 8,074 
White 108 1.5 111 1.9 219 
Two or more 23 0.3 11 0.2 34 
Economic Disadvantage 
Free Lunch 2,188 31.2 1,842 31.9 4,030 
Reduced Lunch 415 5.9 317 5.5 732 
Other Economic Disadvantage 3,965 56.5 3,155 54.7 7,120 
No Economic Disadvantage 451 6.4 459 8.0 910 
Limited English Proficiency (LEP) 
LEP 2,856 40.7 2,291 39.7 5,147 
Not LEP 4,163 59.3 3,482 60.3 7,645 
Total 7,019 100.0 5,773 100.0 12,792 


Notes: Students in grade three are those who repeated the grade; they were enrolled in 
grade three for both the spring 2013 and 2014 STAAR mathematics 
assessments. 

Percentages may not total 100 due to rounding. 

Source: PEIMS Fall Resubmission and TTM files 


HISD Research and Accountability. 


59 


Table 20. Spring 2013 STAAR Mathematics Results for HISD Students Who 
Attended a 2013-2014 Priority or Focus School and Took Both the 


Spring 2013 and 2014 STAAR Mathematics Assessments, by Use of 


TTM 
Did Not Meet Achieved the 
the 2013 Met the 2013 2013 

Phase-In 1 Phase-In 1 Advanced 

Total Standard Standard Standard° 

N N % N N N % 

TTM Users 7,019 3,896 55.5 3,123 44.5 340 48 

Non-TTM Users 5,773 2,941 50.9 2,832 49.1 215 3.7 

Total 12,792 6,837 53.4 5,955 46.6 555 4.3 
Notes: °Students are also included in the number “Met the 2013 Phase-In 1 


Standard.” 


Remaining percentages may not equal 100 due to rounding. 
Sources: 2013 STAAR, 2014 STAAR, 10-8-13 SharePoint listing of 2013-2014 
priority and focus schools, and TTM files 


Table 21. Spring 2014 STAAR Mathematics Results for HISD Students Who 


Attended a 2013-2014 Priority or Focus School and Took Both the 


Spring 2013 and 2014 STAAR Mathematics Assessments, by Use of 


TTM 
Did Not Meet Achieved the 
the 2014 Met the 2014 2014 

Phase-In 1 Phase-In 1 Advanced 

Total Standard Standard Standard° 

N N % N N N % 
TTM Users 7,019 3,234 46.1 3,785 53.9 522 7.4 
Non-TTM Users 5,773 2,541 44.0 3,232 56.0 259 4.4 
Total 12,792 5,775 45.1 7,017 54.9 781 6.1 
Notes: °Students are also included in the number “Met the 2014 Phase-In 1 


Standard.” 


Remaining percentages may not equal 100 due to rounding. 
Sources: 2013 STAAR, 2014 STAAR, 10-8-13 SharePoint listing of 2013-2014 
priority and focus schools, and TTM files 
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Table 22. Propensity Score Matching Results for HISD Students in Priority and Focus 


Schools Who Took Both the Spring 2013 and 2014 STAAR Mathematics 
Assessments, by TTM Usage 


Mean 2014 STAAR Mathematics 
Scale Score 
Non-TTM 
TTM Users Users 
(N=7,019) (N=5,773) Difference S.E. t 
Before Matching 1,529.6 1,567.9 -38.3 2.2 -17.05* 
Matched 1,529.6 1,513.4 16.1 5.2 3.12* 


Note: * indicates p<.001. 


Table 23. Spring 2014 STAAR Mathematics Results for HISD Students Who 
Attended a 2013-2014 Priority or Focus School and Did Not Meet 


the Level 2, Satisfactory, Phase-In 1 Standard on the Spring 2013 
STAAR Mathematics Assessment, by Use of TTM 


Did Not Meet Achieved the 
the 2014 Met the 2014 2014 

Phase-In 1 Phase-In 1 Advanced 

Total Standard Standard Standard° 

N N % N N N % 

TTM Users 3,896 2,733 70.1 1,163 29.9 12 0.3 

Non-TTM Users 2,941 2,023 68.8 918 31.2 7 0.2 

Total 6,837 4,756 69.6 2,081 30.4 19 0.3 
Notes: 


Standard.” 


Sources: 
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Remaining percentages may not equal 100 due to rounding. 
2013 STAAR, 2014 STAAR, 10-8-13 SharePoint listing of 2013-2014 
priority and focus schools, and TTM files 


°Students are also included in the number “Met the 2014 Phase-In 1 
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Table 24. Spring 2014 STAAR Mathematics Results for TTM Users Who Attended a 2013- 
2014 Priority or Focus School and Did Not Meet the Level 2, Satisfactory, Phase- 


In 1 Standard on the Spring 2013 STAAR Mathematics Assessment, by Number 
of TTM Lessons Completed, Number of TTM Lessons Passed, and Number of 
TTM Target Lessons Passed 


Did Not Meet the Met the 2014 Achieved the 
2014 Phase-In 1 Phase-iIn 1 2014 Advanced 
Total Standard Standard Standard° 
N N % N N N % 
TTM Lessons Completed 
0 10 7 70.0 3 30.0 0 0.0 
1-14 2,322 1,667 71.8 655 28.2 5 0.2 
15-29 1,035 704 68.0 331 32.0 4 0.4 
30 or more 529 355 67.1 174 32.9 3 0.6 
TTM Lessons Passed 
0 415 359 86.5 56 13.5 0 0.0 
1-14 3,263 2,282 69.9 981 30.1 9 0.3 
15-29 181 78 43.1 103 56.9 2 1.1 
30 or more 37 14 37.8 23 62.2 1 2.7 
TTM Target Lessons Passed 
0 2,492 1,960 78.7 532 21.3 2 0.1 
1-19 1,394 770 55.2 624 44.8 10 0.7 
20-29 7 3 42.9 4 57.1 0 0.0 
30 or more * * * i * id 
Total 3,896 2,733 70.1 1,163 29.9 12 0.3 
Notes: °Students are also included in the number “Met the 2014 Phase-In 1 Standard.” 


*Results are not provided for fewer than five students. 
Percentages may not total 100 due to rounding. 
Sources: 2013 STAAR, 2014 STAAR, and TTM files 
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Table 25. Regression Analyses of Relationships between 2014 STAAR Mathematics Scale 
Scores and the Number of TTM Lessons Completed, TTM Lessons Passed, and TTM 
Target Lessons Passed by TTM Users Who Attended a Priority or Focus School, Took 


Both the Spring 2013 and 2014 STAAR Mathematics Assessments, and Did Not Meet 


the Level 2, Satisfactory, Phase-In 1 Standard on the Spring 2013 STAAR 
Mathematics Assessment 


Independent Variable Dependent Variable B t p R2 

Number of TTM Lessons 2014 STAAR Mathematics Scale 

-.04 t (3,894) = -2.69 .01 .002 
Completed Score 
Number of TTM Lessons 2014 STAAR Mathematics Scale 

21 t (3,894) = 13.43 < .001 .04 
Passed Score 
Number of TTM Target 2014 STAAR Mathematics Scale 

16 t (3,894) = 10.09 < .001 .03 
Lessons Passed Score 


Note: Effect size conventions for R? are: .01 is small, .09 is medium, and .24 is large. 


Table 26. Number of TTM Lessons Recorded for TTM Users Who Attended a 
Priority or Focus School and Did Not Meet the Spring 2013 STAAR 


Mathematics Level 2, Satisfactory, Phase-In 1 Standard, by Spring 
2014 STAAR Mathematics Performance, 2013-2014 


Number of Lessons 
Standard 
Range Mean Deviation 
Met 2014 Phase-In 1 Standard (N=1,163) 
TTM Lessons Completed 0-276 17.8 21.5 
TTM Lessons Passed 0-108 7.0 8.1 
TTM Target Lessons Passed 0-52 1.8 3.5 
Met 2014 Advanced Standard (N=12) 
TTM Lessons Completed 4-61 22.2 15.4 
TTM Lessons Passed 2-38 11.4 9.9 
TTM Target Lessons Passed 0-17 3.9 5.2 


Sources: 2013 STAAR, 2014 STAAR, and TTM files 
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Table 27. Spring 2014 STAAR Mathematics Results for HISD Students Who 
Attended a 2013-2014 Priority or Focus School and Met the Level 


2, Satisfactory, Phase-In 1 Standard on the Spring 2013 STAAR 


Mathematics Assessment, by Use of TTM 


Did Not Meet Achieved the 
the 2014 Met the 2014 2014 

Phase-In 1 Phase-In 1 Advanced 

Total Standard Standard Standard° 

N N % N N N % 
TTM Users 3,123 501 16.0 2,622 84.0 510 16.3 
Non-TTM Users 2,832 518 18.3 2,314 81.7 252 8.9 
Total 5,955 1,019 17.1 4,936 82.9 762 12.8 
Notes: °Students are also included in the number “Met the 2014 Phase-In 1 Standard.” 


Remaining percentages may not equal 100 due to rounding. 


Sources: 


focus schools, and TTM files 


2013 STAAR, 2014 STAAR, 10-8-13 SharePoint listing of 2013—2014 priority and 


Table 28. Spring 2014 STAAR Mathematics Results for TTM Users Who Attended a 2013- 
2014 Priority or Focus School and Met the Level 2, Satisfactory, Phase-In 1 


Standard on the Spring 2013 STAAR Mathematics Assessment, by Number of 
TTM Lessons Completed, Number of TTM Lessons Passed, and Number of TTM 
Target Lessons Passed 


Did Not Meet the Met the 2014 Achieved the 
2014 Phase-In 1 Phase-iIn 1 2014 Advanced 
Total Standard Standard Standard° 
N N % N N N % 

TTM Lessons Completed 

0 7 4 57.1 3 42.9 1 14.3 

1-14 1,886 355 18.8 1,531 81.2 232 12.3 

15-29 731 99 13.5 632 86.5 133 18.2 

30 or more 499 43 8.6 456 91.4 144 28.9 
TTM Lessons Passed 

0 122 46 37.7 76 62.3 2 1.6 

1-14 2,429 431 17.7 1,998 82.3 288 11.9 

15-29 402 22 5.5 380 94.5 136 33.8 

30 or more 170 2 1.2 168 98.8 84 49.4 
TTM Target Lessons Passed 

0 1,024 306 29.9 718 70.1 29 2.8 

1-19 1,959 194 9.9 1,765 90.1 395 20.1 

20-29 95 1 1.1 94 98.9 59 62.1 

30 or more 45 0 0.0 45 100.0 27 60.0 
Total 3,123 501 16.0 2,622 84.0 510 16.3 

Notes: °Students are also included in the number “Met the 2014 Phase-In 1 Standard.” Percentages may 

not total 100 due to rounding. 
Sources: 2013 STAAR, 2014 STAAR, and TTM files 
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Table 29. Regression Analyses of Relationships between 2014 STAAR Mathematics Scale Scores 
and the Number of TTM Lessons Completed, TTM Lessons Passed, and TTM Target 


Lessons Passed by TTM Users Who Attended a Priority or Focus School, Took Both the 
Spring 2013 and 2014 STAAR Mathematics Assessments, and Met the Level 2, 
Satisfactory, Phase-In 1 Standard on the Spring 2013 STAAR Mathematics Assessment 


Independent Variable Dependent Variable t R2 
Number of TTM Lessons 2014 STAAR Mathematics 
.04 t (3,121) = 2.40 .02 .002 
Completed Scale Score 
Number of TTM Lessons 2014 STAAR Mathematics 
12 t (3,121) = 6.78 < .001 .02 
Passed Scale Score 
Number of TTM Target 2014 STAAR Mathematics 
19 t (3,121) = 10.54 | <.001 .03 
Lessons Passed Scale Score 


Note: Effect size conventions for R? are: .01 is small, .09 is medium, and .24 is large. 


Table 30. Number of TTM Lessons Recorded for TTM Users Who Attended a 
Priority or Focus School and Met the Spring 2013 STAAR 


Mathematics Level 2, Satisfactory, Phase-In 1 Standard, by Spring 
2014 STAAR Mathematics Performance, 2013-2014 


Number of Lessons 
Standard 
Range Mean Deviation 
Met 2014 Phase-In 1 Standard (N=2,622) 
TTM Lessons Completed 0-1,100 18.4 31.4 
TTM Lessons Passed 0-1,050 10.7 25.3 
TTM Target Lessons Passed 0-510 4.8 13.4 
Met 2014 Advanced Standard (N=510) 
TTM Lessons Completed 1-256 24.8 28.0 
TTM Lessons Passed 0-226 17.7 21.0 
TTM Target Lessons Passed 0-149 10.3 14.0 


Sources: 2013 STAAR, 2014 STAAR, and TTM files 
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Table 31. Spring 2014 STAAR Mathematics Results for HISD Students Who Attended a 2013- 


2014 Priority or Focus School and Met the Level 3, Advanced Performance, on the 
Spring 2013 STAAR Mathematics Assessment, by Use of TTM 


HISD Research and Accountability. 


Did Not Meet the Achieved the 2014 

2014 Phase-In 1 Met the 2014 Phase- Advanced 

Total Standard In 1 Standard Standard° 

N N % N N N % 
TTM Users 340 2 0.6 338 99.4 205 60.3 
Non-TTM Users 215 0 0.0 215 100.0 108 50.2 
Total 555 2 0.4 553 99.6 313 56.4 

Notes: °Students are also included in the number “Met the 2014 Phase-In 1 Standard.” 


*Results are not provided for fewer than five students 
Remaining percentages may not equal 100 due to rounding. 

Sources: 2013 STAAR, 2014 STAAR, 10-8-13 SharePoint listing of 2013-2014 priority and 
focus schools, and TTM files 


Table 32. Spring 2014 STAAR Mathematics Results for TTM Users Who Attended a 2013- 
2014 Priority or Focus School and Met the Level 3, Advanced Performance 


Standard, on the Spring 2013 STAAR Mathematics Assessment, by Number of 
TTM Lessons Completed, Number of TTM Lessons Passed, and Number of TTM 
Target Lessons Passed 


Did Not Meet the Met the 2014 Achieved the 
2014 Phase-In 1 Phase-In 1 2014 Advanced 
Total Standard Standard Standard° 
N N % N N N % 

TTM Lessons Completed 

0 * * * * * * * 

1-14 168 2 1.2 166 98.8 88 52.4 

15-29 80 0 0.0 80 100.0 53 66.3 

30 or more 92 0 0.0 92 100.0 64 69.6 
TTM Lessons Passed 

0 * * * * * * * 

1-14 201 2 1.0 199 99.0 103 51.2 

15-29 81 0 0.0 81 100.0 63 77.8 

30 or more 56 0 0.0 56 100.0 38 67.9 
TTM Target Lessons Passed 

0 34 2 5.9 32 94.1 7 20.6 

1-19 244 0 0.0 244 100.0 150 61.5 

20-29 40 0 0.0 40 100.0 32 80.0 

30 or more 22 0 0.0 22 100.0 16 72.7 
Total 340 2 0.6 338 99.4 205 60.3 


Notes: °Students are also included in the number “Met the 2014 Phase-In 1 Standard.” 
*Results are not provided for fewer than five students. 
Percentages may not total 100 due to rounding. 

Sources: 2013 STAAR, 2014 STAAR, and TTM files 
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Table 33. Regression Analyses of Relationships between Spring 2014 STAAR Mathematics Scale 
Scores and the Number of TTM Lessons Completed, TTM Lessons Passed, and TTM 
Target Lessons Passed by TTM Users Who Attended a Priority or Focus School, Took 


Both the Spring 2013 and 2014 STAAR Mathematics Assessments, and Met the Level 
3, Advanced Performance Standard on the Spring 2013 STAAR Mathematics 


Assessment 
Independent Variable Dependent Variable B t p R? 
Number of TTM Lessons 2014 STAAR Mathematics 
-.08 t (338) = -1.56 12 NA 
Completed Scale Score 
Number of TTM Lessons 2014 STAAR Mathematics 
-.07 t (338) = -1.19 24 NA 
Passed Scale Score 
Number of TTM Target 2014 STAAR Mathematics 
: -.02 t (338) = -0.37 71 | NA 
Lessons Passed Scale Score 


Note: Effect size conventions for R? are: .01 is small, .09 is medium, and .24 is large. 


Table 34. Number of TTM Lessons Recorded for TTM Users Who Attended a 
Priority or Focus School and Met the Spring 2013 STAAR 


Mathematics Level 3, Advanced Performance Standard, by Spring 
2014 STAAR Mathematics Performance, 2013-2014 


Number of Lessons 
Standard 
Range Mean Deviation 
Met 2014 Phase-In 1 Standard (N=338) 
TTM Lessons Completed 0-1,100 27.2 67.1 
TTM Lessons Passed 0-1,050 20.8 62.3 
TTM Target Lessons Passed 0-510 12.2 31.6 
Met 2014 Advanced Standard (N=205) 
TTM Lessons Completed 1-256 25.8 29.6 
TTM Lessons Passed 0-226 19.8 24.0 
TTM Target Lessons Passed 0-149 12.7 16.7 


Sources: 2013 STAAR, 2014 STAAR, and TTM files 
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Table 35. Demographic Characteristics of HISD Students Who Retested on STAAR 


Mathematics Assessment, June 2014, by TTM Usage 


TTM Users Non-TTM Users Total 
N % N % N 
School Attended 
Priority School 157 10.6 95 7.2 252 
Focus School 498 33.7 465 35.5 963 
Not a Prioritized School 823 55.7 751 57.3 1,574 
Grade Level 
Grade 5 1,134 76.7 467 35.6 1,601 
Grade 8 344 23.3 844 64.4 1,188 
Gender 
Female 742 50.2 653 49.8 1,395 
Male 736 49.8 658 50.2 1,394 
Race/Ethnicity 
African American 529 35.8 355 274 884 
American Indian 3 0.2 3 0.2 6 
Asian/Pacific Islander 9 0.6 9 0.7 18 
Hispanic 895 60.6 913 69.6 1,808 
White 37 25 28 2.1 65 
Two or more 5 0.3 3 0.2 8 
Economic Disadvantage 
Free Lunch 426 28.8 412 31.4 838 
Reduced Lunch 62 4.2 77 5.9 139 
Other Economic Disadvantage 866 58.6 679 51.8 1,545 
No Economic Disadvantage 124 8.4 143 10.9 267 
Limited English Proficiency (LEP) 
LEP 619 41.9 594 45.3 1,213 
Not LEP 859 58.1 717 54.7 1,576 
Total 1,478 53.0 1,311 47.0 2,789 


Note: *Results are not provided for fewer than five students 
Percentages may not total 100 due to rounding 
Source: PEIMS Fall Resubmission and TTM files 
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Table 36. Spring 2013 STAAR Mathematics Results for HISD Students Who 


Retested on the STAAR Mathematics Assessment, June 2014, by 


Use of TTM 
Did Not Meet Achieved the 
the 2013 Met the 2013 2013 
Phase-In 1 Phase-In 1 Advanced 
Total Standard Standard Standard° 
N N % N N N % 
TTM Users 1,478 1,344 90.9 134 9.1 1 0.1 
Non-TTM Users 1,311 1,130 86.2 181 13.8 1 0.1 
Total 2,789 2,474 _—*88.7 315 11.3 2 0.1 
Notes: °Students are also included in the number “Met the 2013 Phase-In 1 
Standard.” 
Remaining percentages may not equal 100 due to rounding. 
Sources: 2013 STAAR, 2014 STAAR, 10-8-13 SharePoint listing of 2013-2014 
priority and focus schools, and TTM files 
Table 37. 


June 2014 STAAR Mathematics Results for HISD Students Who 


Retested on the STAAR Mathematics Assessment, June 2014, by 


Use of TTM 
Did Not Meet Achieved the 
the 2014 Met the 2014 2014 
Phase-In 1 Phase-In 1 Advanced 
Total Standard Standard Standard° 
N N % N N N % 
TTM Users 1,478 1,099 74.4 379 25.6 0 0.0 
Non-TTM Users 1,311 1,007 76.8 304 23.2 0 0.0 
Total 2,789 2,106 75.5 683 24.5 0 0.0 
Notes: °Students are also included in the number “Met the 2014 Phase-In 1 
Standard.” 
Remaining percentages may not equal 100 due to rounding. 
Sources: 


priority and focus schools, and TTM files 
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2013 STAAR, 2014 STAAR, 10-8-13 SharePoint listing of 2013-2014 
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Table 38. Propensity Score Matching Results for HISD Students Who Took Both the 


Spring 2013 and 2014 STAAR Mathematics Assessments and Retook the 
STAAR Mathematics Assessment in June 2014, by TTM Usage 


Mean 2014 STAAR Mathematics 
Scale Score 
TTM Users Non-TTM Users 
(N=1,478) (N=1,311) Difference S.E. t 
Before Matching 1,421.0 1,478.1 -57.1 2.9 -19.90* 
Matched 1,421.0 1,420.1 0.8 5.7 0.15 


Note: * indicates p<.001. 
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